Welcome, Guest. Please login or register.

Author Topic: Bitmessage with builtin SMTP/POP3 servers  (Read 26688 times)

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #30 on: July 05, 2013, 09:22:23 AM »
I checked Postfix and it replied with 430.  430 is in the same class as 503, according to http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml and since 503 seems to fall more in line with the standard, let's go with that one.
OK

OK, I'll put it on my todo list.  What about just a checkbox on the Inbox page that is something like "Remove E-Mail Headers"?
I would make it E-mail based, since you have the MAIL FROM: field in the SMTP server you know the sender. You can use the from field in the E-Mail client, the user could have an address bitmessage@whatever and mail@whatever and depending on the address you can decide if you want to remove the headers or not.
An alternative would be an additional table in messages.dat, that has the fields "Address" and "Plain". users can add addresses to he table and set them to plaintext (or MIME) and address could be added automatically if they do not send valid MIME mail.

The 'subject' line is part of the inbox SQL table, which allows me to fetch information about each message in the POP3 server without fetching the entire contents of the message.  In the future, this could definitely be improved but I think specially formatted text in either the Subject field or a Bitmessage header would be best.  I'd prefer not to mess with the protocol, so the Bitmessage Subject seems like a good place to put it.
you can use something line SELECT * FROM inbox WHERE INSTR(message,'X-Flags: ')>0 (http://www.sqlite.org/lang_corefunc.html)


Depends on how you look at it.  Your messages weren't properly formatted E-mails, as they didn't contain any E-mail headers. 
The minimum spec is "From" and "Date". I sent two more messages, that match the criteria.
« Last Edit: July 05, 2013, 09:28:57 AM by AyrA »
My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

sarchar

  • Newbie
  • *
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #31 on: July 05, 2013, 09:40:25 AM »
OK, I'll put it on my todo list.  What about just a checkbox on the Inbox page that is something like "Remove E-Mail Headers"?
I would make it E-mail based, since you have the MAIL FROM: field in the SMTP server you know the sender. You can use the from field in the E-Mail client, the user could have an address bitmessage@whatever and mail@whatever and depending on the address you can decide if you want to remove the headers or not.
An alternative would be an additional table in messages.dat, that has the fields "Address" and "Plain". users can add addresses to he table and set them to plaintext (or MIME) and address could be added automatically if they do not send valid MIME mail.
This isn't a bad idea at all. It would allow regular Bitmessage users to pick and choose which addresses are using E-mail clients and which are not. I've added it to my TODO list.

Quote
The 'subject' line is part of the inbox SQL table, which allows me to fetch information about each message in the POP3 server without fetching the entire contents of the message.  In the future, this could definitely be improved but I think specially formatted text in either the Subject field or a Bitmessage header would be best.  I'd prefer not to mess with the protocol, so the Bitmessage Subject seems like a good place to put it.
you can use something line SELECT * FROM inbox WHERE INSTR(message,'X-Flags: ')>0 (http://www.sqlite.org/lang_corefunc.html)
Yeah, this is a possibility. However, it seems to me that it would be better to eventually have a separate boolean column (instead of an expensive string search) so that selects are fast.

Quote
Depends on how you look at it.  Your messages weren't properly formatted E-mails, as they didn't contain any E-mail headers. 
The minimum spec is "From" and "Date". I sent two more messages, that match the criteria.
The Bitmessage Subject needs to meet a few (simple) criteria, too.  I received your last messages. Did you get mine?
« Last Edit: July 05, 2013, 09:45:00 AM by sarchar »
SMTP/POP3 integration with Bitmessage: https://bitmessage.org/forum/index.php/topic,2565.0.html

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #32 on: July 05, 2013, 10:02:33 AM »
This isn't a bad idea at all. It would allow regular Bitmessage users to pick and choose which addresses are using E-mail clients and which are not. I've added it to my TODO list.
But you should allow users to override the automatic settings, so they can force forwarding of certain bitmessage messages to mail and vice versa.
a * Entry would be useful for all unknown addresses, so users can turn on global override of settings.
An address should either be "Mail" "Bitmessage", "Default" or "Mixed"
Mail - Always delivered to E-mail, PyBitmessage appends headers if required on incomming/outgoing messages from this address.
Bitmessage - Opposite of Mail, remove all headers and set the bitmessage subject to the E-mail subject
Default - Uses the value of the global override. This is not applicable to *
Mixed - This address is mixed mode, it sends messages that are valid mails and those which are not. This essentially does not touches messages.


The list should also automatically contain all addresses of the user in "Default" setting so filters can be applied to outgoing and incomming addresses.

Yeah, this is a possibility. However, it seems to me that it would be better to eventually have a separate boolean column (instead of an expensive string search) so that selects are fast.
You receive messages anyway completely from the network, you can then parse them before they are inserted into the database.


The Bitmessage Subject needs to meet a few (simple) criteria, too.  I received your last messages. Did you get mine?
Yes. In the SMTP server, you probably want to change the Content-Encoding to UTF-8 before delivering any messages so users do not need to do so and the setup is automatically correct.
« Last Edit: July 05, 2013, 10:04:12 AM by AyrA »
My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #33 on: July 05, 2013, 10:44:49 AM »
And one more thing.
While it is not in the standard, but specifying AUTH PLAIN without a username (you may test it here telnet://mail.bitmessage.ch:25) should give a "334 Log on" answer and the client specifies the login data after command.
I assume there are Mail clients out there that violate the rfc and thus this "feature" is still included in almost all E-mail servers.

Not required, but nice to have
My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

Atheros

  • Lead Bitmessage developer
  • Administrator
  • Sr. Member
  • *****
  • Posts: 381
  • Karma: +57/-3
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #34 on: July 09, 2013, 02:49:03 PM »
Email address user names are case sensitive. Does anyone know any email clients which don't respect the RFC? BM-asPS98sd9fsDdfGgsoid098ZBsssA@yourcomputer  seems more natural to me.

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #35 on: July 09, 2013, 03:00:46 PM »
As the operator of bitmessage.ch I often see a log entry, about an invalid address being blocked. So there seem to be quite a lot of clients, that do not respect the RFC. I cannot see the messages or the addresses but the log tells E-Mail -> BM direction, so the bm address is wrong.
My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

sarchar

  • Newbie
  • *
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #36 on: July 10, 2013, 06:20:51 AM »
Email address user names are case sensitive. Does anyone know any email clients which don't respect the RFC? BM-asPS98sd9fsDdfGgsoid098ZBsssA@yourcomputer  seems more natural to me.

Personally, I've seen plenty of emails delivered to me with my username fully capitalized.  I don't think there are many E-Mail clients that violate this section of the RFC, however, I do believe there are a lot of crawlers and spammers that don't.  That being said, there's this quote from the Wiki page on E-mail address formatting:

Quote
The local-part "postmaster" is treated specially – it is case-insensitive, and should be forwarded to the server's administrator. Technically all other local-parts are case sensitive, therefore jsmith@example.com and JSmith@example.com specify different mailboxes; however, many organizations treat uppercase and lowercase letters as equivalent.

I also think that @somedomain is quite arbitrary.  Shouldn't it be an address that is clearly distinct from a standard address?

It would be really cool if in the long run, more human-readable addresses were made to work with bitmessage addresses. 

---

Also, there are a number of new features added to my code recently:

* Allow binding to non-127.0.0.1 addresses. This will allow system administrators to provide Bitmessage E-mail accounts on their network.
* Formatting of forwarded mailing list messages.  Mailing list operators will wrap incoming messages with standard email headers if they don't exist already (instead of the "Message ostensibly from..." line).
* Receiving a message at a certain address will, if enabled in settings, automatically convert the message into a E-mail-compliant message. 
* Strip certain headers that might cause a loss of anonymity (User-Agent, Message-ID, etc).  A conversation on the 24x7 list led me to believe this is crucial for adoption.

SMTP/POP3 integration with Bitmessage: https://bitmessage.org/forum/index.php/topic,2565.0.html

Atheros

  • Lead Bitmessage developer
  • Administrator
  • Sr. Member
  • *****
  • Posts: 381
  • Karma: +57/-3
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #37 on: July 11, 2013, 01:10:53 PM »
Email address user names are case sensitive. Does anyone know any email clients which don't respect the RFC? BM-asPS98sd9fsDdfGgsoid098ZBsssA@yourcomputer  seems more natural to me.

Personally, I've seen plenty of emails delivered to me with my username fully capitalized.  I don't think there are many E-Mail clients that violate this section of the RFC, however, I do believe there are a lot of crawlers and spammers that don't.

Indeed, I've gotten email from mass-mailers that change the capitalization of my email address as well. I've also received email at my catch-all email address sent to firstword@example.com when my actual email address was firstword-secondword@example.com. These services are just buggy but that doesn't change whether or not email clients respect capitalization. I say we go with  BM-asPS98sd9fsDdfGgsoid098ZBsssA@whatever

sarchar

  • Newbie
  • *
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #38 on: July 11, 2013, 11:56:35 PM »
Email address user names are case sensitive. Does anyone know any email clients which don't respect the RFC? BM-asPS98sd9fsDdfGgsoid098ZBsssA@yourcomputer  seems more natural to me.

Personally, I've seen plenty of emails delivered to me with my username fully capitalized.  I don't think there are many E-Mail clients that violate this section of the RFC, however, I do believe there are a lot of crawlers and spammers that don't.

Indeed, I've gotten email from mass-mailers that change the capitalization of my email address as well. I've also received email at my catch-all email address sent to firstword@example.com when my actual email address was firstword-secondword@example.com. These services are just buggy but that doesn't change whether or not email clients respect capitalization. I say we go with  BM-asPS98sd9fsDdfGgsoid098ZBsssA@whatever

OK, I'll do the work to change the address format to <BM-ADDR>@something, where the something bit is irrelevant (you can use/specify anything you want).  Does that sound good?  Could also make the domain something unique, @Bitmessage, @BM (BM-xx@BM seems weird though), etc. 
« Last Edit: July 12, 2013, 05:39:00 AM by sarchar »
SMTP/POP3 integration with Bitmessage: https://bitmessage.org/forum/index.php/topic,2565.0.html

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #39 on: July 12, 2013, 05:48:58 AM »
OK, I'll do the work to change the address format to <BM-ADDR>@something, where the something bit is irrelevant (you can use/specify anything you want).  Does that sound good?  Could also make the domain something unique, @Bitmessage, @BM (BM-xx@BM seems weird though), etc.
It is important, that the part after the @ is not used otherwise. I used the trick with bitmessage.ch, since there are only bitmessage addresses registered there. Official invalid TLDs can be found here but they all seem kinda stupid: http://tools.ietf.org/html/rfc2606
My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

r00tz

  • Full Member
  • ***
  • Posts: 111
  • Karma: +3/-2
    • View Profile
    • Green Charter
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #40 on: July 12, 2013, 07:09:04 AM »
I suggest using what is already established as a throw-away, @bm.addr -- it is accepted fine by clients, and thus won't break anonymity further if it does show up somewhere. It is easy and fast to type if it needs to be entered anywhere, and it is easy to remember, after all, it is a bitmessage address.
BM-GtShDbJPw1KSJ46Q93SfCawYe3covKau

AyrA

  • BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1261
  • Karma: +75/-7
  • bitmessage.ch and timeservice operator
    • View Profile
    • AyrAs Homepage
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #41 on: July 12, 2013, 07:22:27 AM »
why do we carry for the part after the @ at all?

My Address: BM-Bc7Rspa4zxAPy9PK26vmcyoovftipStp
Bitmessage Time Service (Subscribe): BM-BcbRqcFFSQUUmXFKsPJgVQPSiFA3Xash
Support the Multipart Message Declaration Draft for Bitmessage: https://bitmessage.org/forum/index.php/topic,1553.0.html
Free Bitmessage to E-Mail Gateway: https://bitmessage.ch

sarchar

  • Newbie
  • *
  • Posts: 26
  • Karma: +0/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #42 on: July 12, 2013, 07:31:22 AM »
My latest implementation doesn't care what comes after the @ in the address, however, when receiving messages a default string will be appended.  In code currently, that is @default, but I can easily change it to @bm.addr and I think it is a decent choice to do so.

Next up, I'm wondering if I should add a new messageEncodingType.  It would be used to signify "this message should be treated as if it *may* contain headers."  Python's email.parser module can be used to parse the content and then check for valid fields ('Date' and 'From'). 

The upside of this is that old-style messages will be delivered just fine.  The downside is that a new style message will appear with "Unknown encoding type.", along with message headers, in your inbox.

Thoughts?
SMTP/POP3 integration with Bitmessage: https://bitmessage.org/forum/index.php/topic,2565.0.html

Justus Ranvier

  • Newbie
  • *
  • Posts: 21
  • Karma: +3/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #43 on: July 18, 2013, 07:03:53 PM »
Is it possible to install and run this version in a headless VM and configure it via text files instead of going through the GUI?

onli

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Bitmessage with builtin SMTP/POP3 servers
« Reply #44 on: August 02, 2013, 03:49:01 PM »
That would indeed be great.

I've got a small bugreport. On my installation, my mail program wasn't able to get any mails from this client. I debugged it and indeed, after the list-command the POP server returned no messages. I debugged it a bit further and the issue seems to be in line 71 of class_pop3Server.py - that if triggers and all messages are continued. I'm not familiar with python so I'm not sure why that occurs. I just commented the line out and it seems to work.