viewpoint-particle

Author Topic: Displaying emoji  (Read 8694 times)

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Displaying emoji
« on: December 10, 2016, 11:34:57 am »
I have a php script that receives inbound SMS messages and writes them into a MySQL database. An emoji in the message (as an example) gets stored as ascii 240 159 152 134. I'm displaying the message using WinBatch and the mshtml com control, and the emoji displays as garbage.

How can I get the emoji to display properly? I googled it and read that I should put <meta charset="utf-8mb"> in the HEAD tag, but that did nothing (or I did it wrong).

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #1 on: December 12, 2016, 06:51:28 am »
Perhaps try 'utf8mb4' which indicates that you need four byte UTF 8 Unicode to render the HTML fully.   Of course, the version of IE on the system running the script will also need to support that character set.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #2 on: December 12, 2016, 08:02:46 am »
I found (in the tech support db) an alternate way of displaying html, namely document.explorer, and the emoji work. But I couldn't get it to work using mshtml.

  • Why would one choose one vs. the other, i.e. mshtml vs. document.explorer?
  • With document.explorer it seems I have to output my html to an external file. Is there any way around that?

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #3 on: December 12, 2016, 09:13:17 am »
Couldn't answer those questions because I have no idea what you mean by 'document.explorer'.   Are you referring to 'Shell.Explorer'?  If so, 'Shell.Explorer' instantiates a WebBrowser object while 'MSHTML:' instantiates just the  MSFT object representing the DOM. 

You would have to ask MSFT the why.

Assuming that you are referring to 'Shell.Explorer', you can  write to the Document object, if you don't wish to load a file or use a URL.

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case @deInit                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
         objBrowser = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)

         If objBrowser.document 
            objBrowser.document.write(:"<HTML><BODY>":@CRLF:"I hate HTML 😁":@CRLF:"</BODY></HTML>")
            objBrowser.document.close()
            objBrowser.document.refresh()
         endif
   endswitch                                                ; Foo_Event
   return(@retDefault)
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"about:blank",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")

"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #4 on: December 12, 2016, 12:42:01 pm »
Thanks for intuiting right re Shell.Explorer.

Thanks for that example... sorry, I should have mentioned I'm still on format 6.1.

When I try

         objBrowser = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)

         If objBrowser.document 

the first statement returns a handle-like number, but the second statement returns a FALSE.

Mine is:

      objBrowser = DialogObject(Schmandle, 18, 3)
      If objBrowser.document 
    objBrowser.document.write(FullThread)
         objBrowser.document.close()
         objBrowser.document.refresh()
      endif

"document" is defined elsewhere?

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #5 on: December 12, 2016, 01:40:31 pm »
In the the example above the "about:blank" URL in the dialog template will cause the WebBrowser control to create a  document object that you can then access using the 'document' property of Webbrowser object. Do you have that or something like it in your dialog template?
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #6 on: December 12, 2016, 01:52:26 pm »
Nope...

SMSDialog018=`3,147,172,154,COMCONTROL,DEFAULT,"Shell.Explorer.2",default,99,DEFAULT,DEFAULT,DEFAULT,DEFAULT`


td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #7 on: December 12, 2016, 02:00:03 pm »
if you put  "about:blank" in place of "Shell.Explorer.2", you should get a WebBrowser object as represented by the progid "Shell.Explorer.2" with a document object and not have to call the navigate method with the "about:blank" URL to create a document.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #8 on: December 12, 2016, 02:16:27 pm »
That worked. Sort of. What shows up in my dialog now is a white rectangle, no text. I had hoped to see the contents of the FullThread variable.

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #9 on: December 12, 2016, 02:37:44 pm »
The HTML needs to be well formed but if it worked using MSHTML: it should work with Shell.Explorer.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #10 on: December 12, 2016, 03:30:46 pm »
It did work with Shell.Explorer...

document = DialogObject(Schmandle, 18, 3)
document.navigate("About:Blank",0,'','','')
document.navigate(FullThreadFile,0,"","","")

I was simply using a put command to stuff FullThread into FullThreadFile.

CORRECTION: Actually, it looks like it is working... I just need to find out why my FullThread variable isn't initializing properly.

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #11 on: December 12, 2016, 05:13:22 pm »
Current status... out of the three methods of displaying the html, the only one that displays the emoji is when I output the html to a temp file, then load that file using my document.navigate command.

(sample file attached)

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #12 on: December 13, 2016, 09:25:57 am »
If you want to use a variable instead of a file, you need to tell WinBatch how to convert the text in the variable containing the html to UTF-16.  To do this you need to set the code page to UTF-8:

nCodePage = ChrSetCodePage(65001)

And once you are done in your dialog, you need to set it back to the default:

ChrSetCodePage(nCodePage)

I don't know how exactly you are assigning the html to a variable so I can't tell you exactly where to change the code page.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #13 on: December 13, 2016, 10:31:36 am »
Another example:

Code: Winbatch
; Since RAW utf-8 cannot be embedded as a sting constant without converting
; it to a hex string, we use FileGet(W) to simulate some kind of GET operation.
;;;
; Would need to perform the conversion before the write statement with this
; approach.
;strHtml = FileGet(DirScript():'FullThreadFile.html')
; Variable now contains the raw HTML with a BOM so
;; remove the BOM.
;strHtml = StrSub(strHtml, 4, -1)
;;;
;; Read Unicode into a variable using correct code page.
nCodePage = ChrSetCodePage(65001)
strHtml = FileGetW(DirScript():'FullThreadFile.html')
ChrSetCodePage(nCodePage)
; BOM squad.
strHtml = StrSub(strHtml, 2, -1)

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case @deInit                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
         objDocument = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)
         
         If objDocument
           ; If the conversion takes place here, need
           ; to use ChrSetCodePaege.
           ;nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ;ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return(@retDefault)
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")
exit
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #14 on: December 13, 2016, 12:14:16 pm »
This solution would seem to rely on a file read and write, i.e. FILEGETW. I was hoping to do this without disk interaction. Am I reading it wrong?

Presently I'm putting FullThread in to FullThreadFile.html, then displaying it using:

document = DialogObject(Schmandle, 18, 3)
document.navigate("About:Blank",0,'','','')
document.navigate(FullThreadFile,0,"","","")

This happens a lot while the script runs; I was hoping to avoid the disk access. But this method does render the emoji.

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #15 on: December 13, 2016, 01:58:44 pm »
Yes, you are reading it wrong.  You need to look at the last example carefully and read the embedded comments. 

Your problem revolves around the fact that an COM Automation is UTF-16 Unicode based so almost all text passed to a COM Automation method is converted to UTF-16 Unicode.   You have text that is UTF-8 but WinBatch does not know that so it converts the text to UTF-16 using the current user's code page.  The purpose of calling ChrSetCodePage(65001) is to tell WinBatch to use the UTF-8 code page when converting the HTML to UTF-16.   Since the file you attached to a previous post is in UTF-8 you must have the HTML you want to display in UTF-8 as some point in the process. 

Basically, if you can put that UTF-8 HTML into a WIL variable without using an intermediate file then you can display it in a WIL dialog COMCONTROL with the help of the ChrSetCodePage function.

"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #16 on: December 13, 2016, 02:33:22 pm »
One last example:

Code: Winbatch
#DefineFunction MagicHappens()
   strHtml =  "<!DOCTYPE html>"
   strHtml :=  '<html><head><meta http-equiv="Content-Type" content="text/html;charset="utf-8mb4"></head>'  
   strHtml :=  "<body>"
   strHtml :=  "<p style=color:#0000ff;font-family:Arial;font-size:11px;font-weight:bold;text-align:left;> <span style=background-color:#FFFFA3> 12/12/2016, 8:09pm,  babe said:    <br> <span style=font-weight:normal;font-style:italic;background-color:#ece9d8> Fred 😰👁💋 </p>"
   strHtml :=  "</body>"  
   strHtml :=  "</html>"
   return strHtml
#EndFunction

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case @deInit                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
         objDocument = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)
         
         If objDocument
           ; If the conversion takes place here, need
           ; use ChrSetCodePage.
           strHtml = MagicHappens()
           nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return(@retDefault)
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")
exit
 
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #17 on: December 14, 2016, 12:48:27 pm »
Reckon I need to upgrade... chrSetCodePath doesn't seem to exist in my WinBatch!

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #18 on: December 15, 2016, 06:51:11 am »
That's 'ChrSetCodePage' and not 'ChrSetCodePath'.  You might be able to use a couple of DllCalls to convert your text but I am not sure that you could get it passed to the COM Automation method in a version of WinBatch that is so old it does not have the ChrSetCodePage function.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #19 on: December 15, 2016, 07:30:10 am »
OK; I'll try again. You said ChrSetCodePath a couple of times. Just a typo I now know.

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #20 on: December 15, 2016, 12:43:49 pm »
This example works. I adapted it for 6.1.

The method for doing this, in summary, is:

  • Include the charset="utf-8mb4" meta tag in the html
  • Assemble the html using default encoding
  • Turn on the 65001 codepage just before writing
  • Turn default codepage back on after writing


Code: Winbatch
#DefineFunction MagicHappens()
   strHtml =  "<!DOCTYPE html>"
   strHtml= strHtml : '<html><head><meta http-equiv="Content-Type" content="text/html";charset="utf-8mb4"></head>'  
   strHtml= strHtml : "<body>"
   strHtml= strHtml : "<p style=color:#0000ff;font-family:Arial;font-size:11px;font-weight:bold;text-align:left;> <span style=background-color:#FFFFA3> 12/12/2016, 8:09pm,  babe said:    <br> <span style=font-weight:normal;font-style:italic;background-color:#ece9d8> Fred 😰👁💋 </p>"
   strHtml= strHtml : "</body>"  
   strHtml= strHtml : "</html>"
   return strHtml
#EndFunction

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case 0                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,2,@TRUE)
         objDocument = DialogObject(Foo_Handle, 3, 3)
         
         If objDocument
           ; If the conversion takes place here, need
           ; use ChrSetCodePath.
           strHtml = MagicHappens()
           nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return -1
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.1`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,    DEFAULT,"OK",    1,10,default,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,    PUSHBUTTON,DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,    COMCONTROL,default,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`


ButtonPushed=Dialog("Foo")
exit

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #21 on: December 15, 2016, 01:18:07 pm »
OK; I'll try again. You said ChrSetCodePath a couple of times. Just a typo I now know.

The typo 'ChrSetCodePath' appeared in a code comment twice.  Whereas the correct term 'ChrSetCodePage' appeared as uncommented code in several examples and in all the posted discussion. 

I a bit perplexed why you would pick out the two comments instead of following the actual syntax highlighted code in the examples and the discussion text.   Or at least asked for clearification.
"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

stevengraff

  • Sr. Member
  • ****
  • Posts: 271
Re: Displaying emoji
« Reply #22 on: December 15, 2016, 01:38:55 pm »
I focused on the chrSetCodePath because I was having zilcho success with chrSetCodePage and assumed a) you did not make a mistake, and b) my lack of using chrSetCodePath could be the problem.

td

  • Tech Support
  • *****
  • Posts: 3985
    • WinBatch
Re: Displaying emoji
« Reply #23 on: December 15, 2016, 01:57:12 pm »
For ye have the typo with you always.

"No one who sees a peregrine falcon fly can ever forget the beauty and thrill of that flight."
  - Dr. Tom Cade

kdmoyers

  • Sr. Member
  • ****
  • Posts: 448
Re: Displaying emoji
« Reply #24 on: December 16, 2016, 05:57:26 am »
And Loki saw the Men were prideful, and so gave them Typo,
So they should always remember they are made of clay.
The mind is everything; What you think, you become.