Displaying emoji

stevengraff · December 10, 2016, 11:34:57 AM

I have a php script that receives inbound SMS messages and writes them into a MySQL database. An emoji in the message (as an example) gets stored as ascii 240 159 152 134. I'm displaying the message using WinBatch and the mshtml com control, and the emoji displays as garbage.

How can I get the emoji to display properly? I googled it and read that I should put <meta charset="utf-8mb"> in the HEAD tag, but that did nothing (or I did it wrong).

td · December 12, 2016, 06:51:28 AM

Perhaps try 'utf8mb4' which indicates that you need four byte UTF 8 Unicode to render the HTML fully. Of course, the version of IE on the system running the script will also need to support that character set.

stevengraff · December 12, 2016, 08:02:46 AM

I found (in the tech support db) an alternate way of displaying html, namely document.explorer, and the emoji work. But I couldn't get it to work using mshtml.

Why would one choose one vs. the other, i.e. mshtml vs. document.explorer?
With document.explorer it seems I have to output my html to an external file. Is there any way around that?

td · December 12, 2016, 09:13:17 AM

Couldn't answer those questions because I have no idea what you mean by 'document.explorer'. Are you referring to 'Shell.Explorer'? If so, 'Shell.Explorer' instantiates a WebBrowser object while 'MSHTML:' instantiates just the MSFT object representing the DOM.

You would have to ask MSFT the why.

Assuming that you are referring to 'Shell.Explorer', you can write to the Document object, if you don't wish to load a file or use a URL.

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
switch Foo_Event ; Switch based on Dialog Message type
case @deInit ; Standard Initialization message
DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
objBrowser = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)

If objBrowser.document
objBrowser.document.write(:"<HTML><BODY>":@CRLF:"I hate HTML 😁":@CRLF:"</BODY></HTML>")
objBrowser.document.close()
objBrowser.document.refresh()
endif
endswitch ; Foo_Event
return(@retDefault)
#EndSubroutine ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"about:blank",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")

stevengraff · December 12, 2016, 12:42:01 PM

Thanks for intuiting right re Shell.Explorer.

Thanks for that example... sorry, I should have mentioned I'm still on format 6.1.

When I try

objBrowser = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)

If objBrowser.document

the first statement returns a handle-like number, but the second statement returns a FALSE.

Mine is:

objBrowser = DialogObject(Schmandle, 18, 3)
If objBrowser.document
objBrowser.document.write(FullThread)
objBrowser.document.close()
objBrowser.document.refresh()
endif

"document" is defined elsewhere?

td · December 12, 2016, 01:40:31 PM

In the the example above the "about:blank" URL in the dialog template will cause the WebBrowser control to create a document object that you can then access using the 'document' property of Webbrowser object. Do you have that or something like it in your dialog template?

stevengraff · December 12, 2016, 01:52:26 PM

Nope...

SMSDialog018=`3,147,172,154,COMCONTROL,DEFAULT,"Shell.Explorer.2",default,99,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

td · December 12, 2016, 02:00:03 PM

if you put "about:blank" in place of "Shell.Explorer.2", you should get a WebBrowser object as represented by the progid "Shell.Explorer.2" with a document object and not have to call the navigate method with the "about:blank" URL to create a document.

stevengraff · December 12, 2016, 02:16:27 PM

That worked. Sort of. What shows up in my dialog now is a white rectangle, no text. I had hoped to see the contents of the FullThread variable.

td · December 12, 2016, 02:37:44 PM

The HTML needs to be well formed but if it worked using MSHTML: it should work with Shell.Explorer.

stevengraff · December 12, 2016, 03:30:46 PM

It did work with Shell.Explorer...

document = DialogObject(Schmandle, 18, 3)
document.navigate("About:Blank",0,'','','')
document.navigate(FullThreadFile,0,"","","")

I was simply using a put command to stuff FullThread into FullThreadFile.

CORRECTION: Actually, it looks like it is working... I just need to find out why my FullThread variable isn't initializing properly.

stevengraff · December 12, 2016, 05:13:22 PM

Current status... out of the three methods of displaying the html, the only one that displays the emoji is when I output the html to a temp file, then load that file using my document.navigate command.

(sample file attached)

td · December 13, 2016, 09:25:57 AM

If you want to use a variable instead of a file, you need to tell WinBatch how to convert the text in the variable containing the html to UTF-16. To do this you need to set the code page to UTF-8:

nCodePage = ChrSetCodePage(65001)

And once you are done in your dialog, you need to set it back to the default:

ChrSetCodePage(nCodePage)

I don't know how exactly you are assigning the html to a variable so I can't tell you exactly where to change the code page.

td · December 13, 2016, 10:31:36 AM

Another example:

Code (winbatch) Select

; Since RAW utf-8 cannot be embedded as a sting constant without converting 
; it to a hex string, we use FileGet(W) to simulate some kind of GET operation.
;;;
; Would need to perform the conversion before the write statement with this
; approach.
;strHtml = FileGet(DirScript():'FullThreadFile.html')
; Variable now contains the raw HTML with a BOM so
;; remove the BOM.
;strHtml = StrSub(strHtml, 4, -1)
;;;
;; Read Unicode into a variable using correct code page. 
nCodePage = ChrSetCodePage(65001)
strHtml = FileGetW(DirScript():'FullThreadFile.html')
ChrSetCodePage(nCodePage)
; BOM squad.
strHtml = StrSub(strHtml, 2, -1)

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case @deInit                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
         objDocument = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)
         
         If objDocument
           ; If the conversion takes place here, need
           ; to use ChrSetCodePaege.
           ;nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ;ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return(@retDefault)
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")
exit

stevengraff · December 13, 2016, 12:14:16 PM

This solution would seem to rely on a file read and write, i.e. FILEGETW. I was hoping to do this without disk interaction. Am I reading it wrong?

Presently I'm putting FullThread in to FullThreadFile.html, then displaying it using:

document = DialogObject(Schmandle, 18, 3)
document.navigate("About:Blank",0,'','','')
document.navigate(FullThreadFile,0,"","","")

This happens a lot while the script runs; I was hoping to avoid the disk access. But this method does render the emoji.

td · December 13, 2016, 01:58:44 PM

Yes, you are reading it wrong. You need to look at the last example carefully and read the embedded comments.

Your problem revolves around the fact that an COM Automation is UTF-16 Unicode based so almost all text passed to a COM Automation method is converted to UTF-16 Unicode. You have text that is UTF-8 but WinBatch does not know that so it converts the text to UTF-16 using the current user's code page. The purpose of calling ChrSetCodePage(65001) is to tell WinBatch to use the UTF-8 code page when converting the HTML to UTF-16. Since the file you attached to a previous post is in UTF-8 you must have the HTML you want to display in UTF-8 as some point in the process.

Basically, if you can put that UTF-8 HTML into a WIL variable without using an intermediate file then you can display it in a WIL dialog COMCONTROL with the help of the ChrSetCodePage function.

td · December 13, 2016, 02:33:22 PM

One last example:

Code (winbatch) Select

#DefineFunction MagicHappens()
   strHtml =  "<!DOCTYPE html>" 
   strHtml :=  '<html><head><meta http-equiv="Content-Type" content="text/html;charset="utf-8mb4"></head>'  
   strHtml :=  "<body>" 
   strHtml :=  "<p style=color:#0000ff;font-family:Arial;font-size:11px;font-weight:bold;text-align:left;> <span style=background-color:#FFFFA3> 12/12/2016, 8:09pm,  babe said:    <br> <span style=font-weight:normal;font-style:italic;background-color:#ece9d8> Fred ÃƒÂ°Ã...Â¸Ã‹Å"Ã,Â°ÃƒÂ°Ã...Â¸Ã¢â,¬ËœÃ,ÂÃƒÂ°Ã...Â¸Ã¢â,¬â,,¢Ã¢â,¬Â¹ </p>" 
   strHtml :=  "</body>"  
   strHtml :=  "</html>"
   return strHtml
#EndFunction

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case @deInit                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,@dePbPush,@TRUE)
         objDocument = DialogObject(Foo_Handle, "ComControl_1", @doGetObject)
         
         If objDocument
           ; If the conversion takes place here, need
           ; use ChrSetCodePage.
           strHtml = MagicHappens()
           nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return(@retDefault)
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.2`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,"PushButton_OK",DEFAULT,"OK",1,10,@csDefButton,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,PUSHBUTTON,"PushButton_Cancel",DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,COMCONTROL,"ComControl_1",DEFAULT,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("Foo")
exit

stevengraff · December 14, 2016, 12:48:27 PM

Reckon I need to upgrade... chrSetCodePath doesn't seem to exist in my WinBatch!

td · December 15, 2016, 06:51:11 AM

That's 'ChrSetCodePage' and not 'ChrSetCodePath'. You might be able to use a couple of DllCalls to convert your text but I am not sure that you could get it passed to the COM Automation method in a version of WinBatch that is so old it does not have the ChrSetCodePage function.

stevengraff · December 15, 2016, 07:30:10 AM

OK; I'll try again. You said ChrSetCodePath a couple of times. Just a typo I now know.

stevengraff · December 15, 2016, 12:43:49 PM

This example works. I adapted it for 6.1.

The method for doing this, in summary, is:

Include the charset="utf-8mb4" meta tag in the html
Assemble the html using default encoding
Turn on the 65001 codepage just before writing
Turn default codepage back on after writing

Code (winbatch) Select


#DefineFunction MagicHappens()
   strHtml =  "<!DOCTYPE html>" 
   strHtml= strHtml : '<html><head><meta http-equiv="Content-Type" content="text/html";charset="utf-8mb4"></head>'  
   strHtml= strHtml : "<body>" 
   strHtml= strHtml : "<p style=color:#0000ff;font-family:Arial;font-size:11px;font-weight:bold;text-align:left;> <span style=background-color:#FFFFA3> 12/12/2016, 8:09pm,  babe said:    <br> <span style=font-weight:normal;font-style:italic;background-color:#ece9d8> Fred ÃƒÂ°Ã...Â¸Ã‹Å"Ã,Â°ÃƒÂ°Ã...Â¸Ã¢â,¬ËœÃ,ÂÃƒÂ°Ã...Â¸Ã¢â,¬â,,¢Ã¢â,¬Â¹ </p>" 
   strHtml= strHtml : "</body>"  
   strHtml= strHtml : "</html>"
   return strHtml
#EndFunction

#DefineSubroutine FooBar (Foo_Handle,Foo_Event,Foo_Name,Foo_EventInfo,Foo_ChangeInfo)
   switch Foo_Event                                         ; Switch based on Dialog Message type
      case 0                                          ; Standard Initialization message
         DialogProcOptions(Foo_Handle,2,@TRUE)
         objDocument = DialogObject(Foo_Handle, 3, 3)
         
         If objDocument
           ; If the conversion takes place here, need
           ; use ChrSetCodePath.
           strHtml = MagicHappens()
           nCodePage = ChrSetCodePage(65001)
           objDocument.write(strHtml)
           ChrSetCodePage(nCodePage)
           objDocument.close()
           objDocument.refresh()
         endif
   endswitch                                                ; Foo_Event
   return -1
#EndSubroutine                                              ; End of Dialog Callback FooCallbackProc

FooFormat=`WWWDLGED,6.1`

FooCaption=`Foo Example`
FooX=860
FooY=075
FooWidth=184
FooHeight=166
FooNumControls=003
FooProcedure=`FooBar`
FooFont=`DEFAULT`
FooTextColor=`DEFAULT`
FooBackground=`DEFAULT,DEFAULT`
FooConfig=0

Foo001=`014,138,033,011,PUSHBUTTON,    DEFAULT,"OK",    1,10,default,DEFAULT,DEFAULT,DEFAULT`
Foo002=`129,138,033,011,    PUSHBUTTON,DEFAULT,"Cancel",0,20,DEFAULT,DEFAULT,DEFAULT,DEFAULT`
Foo003=`007,009,157,116,    COMCONTROL,default,"MSHTML:",DEFAULT,30,DEFAULT,DEFAULT,DEFAULT,DEFAULT`


ButtonPushed=Dialog("Foo")
exit

td · December 15, 2016, 01:18:07 PM

Quote from: stevengraff on December 15, 2016, 07:30:10 AM
OK; I'll try again. You said ChrSetCodePath a couple of times. Just a typo I now know.

The typo 'ChrSetCodePath' appeared in a code comment twice. Whereas the correct term 'ChrSetCodePage' appeared as uncommented code in several examples and in all the posted discussion.

I a bit perplexed why you would pick out the two comments instead of following the actual syntax highlighted code in the examples and the discussion text. Or at least asked for clearification.

stevengraff · December 15, 2016, 01:38:55 PM

I focused on the chrSetCodePath because I was having zilcho success with chrSetCodePage and assumed a) you did not make a mistake, and b) my lack of using chrSetCodePath could be the problem.

td · December 15, 2016, 01:57:12 PM

For ye have the typo with you always.

kdmoyers · December 16, 2016, 05:57:26 AM

And Loki saw the Men were prideful, and so gave them Typo,
So they should always remember they are made of clay.

News:

Displaying emoji