WordPress.org

Ready to get started?Download WordPress

Forums

Events Manager
[resolved] Notification emails display & instead of & (24 posts)

  1. Daedalon
    Member
    Posted 1 year ago #

    1. In the notification emails I see "&amp"; instead of "&".

    My guess is that htmlspecialchars() is called one time too many.

    2. Since this is not worth opening a separate topic on its own, I'd like to suggest that the Event Submitted template would default to showing "View this event - #_EVENTURL" before "Edit this event" instead of the current reverse order. I've changed ours to do this so the change to the defaults wouldn't affect our site, but it might help all future installations, so wanted to bring this to your consideration.

    http://wordpress.org/extend/plugins/events-manager/

  2. agelonwl
    Member
    Posted 1 year ago #

    1. Did you enable Settings > Emails > Email Settings > Send HTML Emails and/or try changing mail sending method?

  3. Daedalon
    Member
    Posted 1 year ago #

    1. No, the emails are sent as plaintext. This happens with PHP mail function, Sendmail, Qmail and WP Mail sending methods, everything except SMTP which we don't use. These methods shouldn't have anything to do with htmlspecialchars conversions.

    Digging into the code I found that the $subject and $body parameters that are used to call EM_Mailer::send() via EM_Event->email_send() are already formatted wrong at the time of that call. The fix can be applied at em-emails.php by locating the lines 47-49:

    $subject = $EM_Event->output(get_option('dbem_event_published_email_subject'), 'raw');
    	        	$body = $EM_Event->output(get_option('dbem_event_published_email_body'), $output_type);
    		        $EM_Event->email_send( $subject, $body, $admin_emails);

    and adding four lines, in effect changing them to

    $subject = $EM_Event->output(get_option('dbem_event_published_email_subject'), 'raw');
    	        	$body = $EM_Event->output(get_option('dbem_event_published_email_body'), $output_type);
    				if ( $output_type != "html" ) {
    					$subject = htmlspecialchars_decode( $subject );
    					$body = htmlspecialchars_decode( $body );
    				}
    		        $EM_Event->email_send( $subject, $body, $admin_emails);

    The same four-line if should also be inserted right above the $EM_Event->email_send() call on lines 41 and 26, after renaming $message to $body on lines 41, 38 and 35 (I thought that renaming was done a month ago already). These two make sure the also other notification emails get the proper formatting.

    Possibly an even more proper fix is to make classes/em-event.php's function output() apply htmlspecialchars_decode() always when it is called with the parameter used for plaintext emails.

  4. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    will test/fix

    2. we'll be revising the templates in the future but right now to save more work for translators we will leave as is and do it in one go ina few months.

  5. Daedalon
    Member
    Posted 1 year ago #

    1. Great!

    2. Sounds good as well.

  6. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    btw, Ive tested this on a shared server using phpmail and I can't reproduce this. are you on the latest versions of EM? We made some adjustments recently to email sending and html formatting.

    Does it happen also for booking emails e.g. confirmations?

  7. Daedalon
    Member
    Posted 1 year ago #

    This has happened with all recent EM version. Using latest 5.3 now.

    The main reason seems to be that the event titles are in the database in a htmlencoded format. I've had to add a check to templates/forms/event-editor.php to filter $EM_Event->event_name always through html_entity_decode(), otherwise the input field gets populated with "&" instead of "&" and so forth. Html_entity_decode() works better instead of htmlspecialchars_decode() that I used in the patch above, as also characters such as "@" are affected.

    I don't know why the event_name field is in the db in an encoded format but all the others are in a raw format, but this has required to filter the data through html_entity_decode every time it's retrieved from db and put into a field where all the characters shouldn't be encoded (eg. " yes, @ not).

    If you have the event_name field content in the db in raw format, eg. "&" instead of "&", then it must be something on our end only.

  8. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    ah... maybe I didn't check this way, I just modified the email template to test it out.

    so the problem then is if you create an event with a title like & and your email uses #_EVENTNAME?

    If you have any example data (e.g. event title or any other info reproducing your situation) that I can use to test that'd be nice. I'm going to release 5.3.1 now though so it'll have to wait until next release (or earlier a dev version)

  9. Daedalon
    Member
    Posted 1 year ago #

    Yes, more or less. Recap for clarity:

    1. User creates an event with "&" in the title.
      -> The database entry has "&" instead of "&" in the title.
    2. Email notifications were on.
      -> The notification email shows "&" in the event title instead of "&".
      - This happens because the event_name data from db was in encoded format and it was sent in the email without decoding it first.
    3. User visits the front-end edit event page, which uses the default template.
      -> Event title field shows "&" instead of "&".
      - This happens because the event_name data from db was in encoded format and templates/forms/event-editor.php encodes it yet again: htmlspecialchars($EM_Event->event_name,ENT_QUOTES)

    Generally speaking, depending on the context, the output needs to be encoded a) only via htmlspecialchars( ENT_QUOTES ), or b) more fully via html_entity_decode() or c) not at all. Now the only way to write secure and functional front-end code is to first decode event_name and then, if needed, re-encode it.

    Doing so is a better choice than having "&" be displayed to users, so I hope that EM core implements either:

    • this workaround, so that in all places where EM_Event->event_name is used, it is decoded (as needed in emails and templates) and then encoded in the proper way if needed (inside event-editor.php's < input >), or
    • the long-term solution of making sure all event data, including event_name, is written to database in a raw, nonencoded format. When implementing this, a certain amount of care is needed to convert the existing event data just once, regardless of from which to which version the user is updating.
  10. Daedalon
    Member
    Posted 1 year ago #

    As can be seen above, wordpress.org's BBpress is also overly aggressive in converting & amp; (without space) to &. The first (or only) "&" on any line except "1. User creates...", should be read as "& amp;" (without space).

    The issue persists in EM 5.3.1.

  11. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    noted down, will look into this

  12. Daedalon
    Member
    Posted 1 year ago #

    Another workaround would be to have EM_Events::get() and related functions always run event title through html_entity_decode() before passing it onwards.

  13. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    btw, this should be fixed in 5.3.5

  14. Daedalon
    Member
    Posted 1 year ago #

    Excellent, will test.

  15. Daedalon
    Member
    Posted 1 year ago #

    The old issue with HTML entities being encoded in notification emails still persists in 5.3.8.1-dev. Submitting an event with a title of "&" results in an email notification showing the title as "& amp;".

  16. agelonwl
    Member
    Posted 1 year ago #

    hi,

    thanks, am going to test this out and inform Marcus about it incase that this is a known bug.

  17. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    what about 5.3.9?

    i don't think it's a problem with emails themselves, but how the front-end submission form dealt with entities, which I think is sorted now

  18. Daedalon
    Member
    Posted 1 year ago #

    Still happened with 5.3.9 and a non-modified forms/event-editor.php. & is shown as & amp;, < is shown as & lt; and so forth in notification email. Didn't notice this before, but even the edit event link is wrong: [siteurl]/[editevent]/?action=edit& #038;event_id=1234 (without the space). Shouldn't have that "#038;".

  19. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    I'll have another look, but I remember this being fixed, so can you re-confirm reproducing it this way:

    sending mail as html is disabled (so plaintext)
    you're using #_EVENTURL or #_EVENTNAME and that's converting any relevant characters into html entities?

  20. Daedalon
    Member
    Posted 1 year ago #

    Steps taken on a 5.3.9:

    1. Checked email sending settings: HTML is disabled (emails are plaintext).

    2. Renamed the template folder, so only default templates are used.

    3. Opened the page for submitting an event.
    Changed name to: & <
    Changed start date.
    Changed location to an already existing one. Submitted. Event got automatically published.

    4. Received email notification with the subject: "Published Event - & amp; & lt;" (without the two spaces).

    A line from the message body: "Name: & amp; & lt;" (without the two spaces). This part comes from #_EVENTNAME.

    Checked database. The name of this event is in database as "& <", exactly as it should. Some events have "& amp;" in their event_name due to pre-5.3.9 code. It seems that between retrieval from database and the sending of email the characters are HTML encoded even when the email is plaintext.

  21. Daedalon
    Member
    Posted 1 year ago #

    Tried also with HTML emails. In them the subject is incorrectly encoded, but body is shown properly. Mails are sent using PHP mail function.

    Also noticed that having a < and then > removes both and everything in between them. This can be seen both by submitting an event without a date and looking at the name field, or submitting an event successfully and looking at the title. Publishing "& <123> >" results in an email subject "Published Event - & amp;  & gt;" and a database field event_name of "&  >".

  22. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    I've been spending (too much) time testing this nuance, and it turns out that this is how WP behaves.

    Try making a test user an author and try submitting this post title:

    Test Post ' & >< - " '

    Only Editors/Admins get away with this.

    I don't think it's worth risking converting entities in the message of emails for these special cases (that's why you can turn on HTML emails), but I do agree with the subject needing decoding since that is always plaintext, so that'll be done next update.

  23. Daedalon
    Member
    Posted 1 year ago #

    That's good news about the subject line, that has been the most noticeable thing. Thanks for looking into this.

    Wouldn't it be possible to esc_attr() the non-HTML body and then unencode it? This would have a small performance hit on the server when sending an email, but the emails would always be properly (un)encoded.

    HTML emails consume more bandwidth and aren't as readable on all programs, so we'd rather enable that only for those emails that need rich formatting: images, tables, font colors and sizes etc. Odd that WordPress encodes the non-HTML email bodies for non-Editors/Admins. If I'd know more about the specifics I'd see about opening a bug in core Trac.

  24. Marcus
    NetWebLogic Support
    Plugin Author

    Posted 1 year ago #

    Wouldn't it be possible to esc_attr() the non-HTML body and then unencode it?

    Given you could potentially reencode bad html i wouldn't take that risk... but you can :)

    probably the easiest way is creating a custom class which extends our EM_Mailer, overwrite that global $EM_Mailer variable, and then add a send function which does your filtering first.

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic

Tags

No tags yet.