Quiz for Duck

[stanl]

A few threads ago when a discussion came up about encryption the old Caesar Cipher was raised. I played around with it to encrypt a user, password, passphrase into a file: [attatched] but with a simple clue as to how to decypher.  you need to come up with


u = SmileyDan
p=  CuriousMelba2$
f = c:\temp\pf.sys


just playing with hash table / map === but think it would keep curious onlookers away

[td]

Gaius Julius would be proud.

[stanl]

But, there is a problem  While PS can use Import-Csv to obtain the file contents, attempting

Code (WINBATCH) Select Expand



file = "C:\temp\testhash.sys"
If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
map = MapFileGetCSV(file)



will error that file contains null characters.

[td]

Yup. WIL does not allow null characters in ANSI CSV files for obvious reasons. I guess you will have to settle for  PS.

[stanl]

Below seems to work. Picking a Random number for initial offset but including it in output file.  Point being multiple runs will/should have different results in the output file but with same 'data' if offset is known. Oh, the code is clumsy and could have used a couple of UDF's, but it illustrates something I need to show somebody. As for the initial failure with Nulls.... I was initially testing in PS and tested PS output with WB.... dunno.   

Code (WINBATCH) Select Expand



file = "C:\temp\testhash.sys"
If fileexist(file) Then filedelete(file)
offset = Random((255 -122)/2)
text = "i,v":@CRLF
text = text:"ii,":offset:@CRLF:"u,"
var = "SmileyDan"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
text = text:@CRLF:"p,"
var = "CuriousMelba2$"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
text = text:@CRLF:"f,"
var = "c:\temp\pf.sys"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
Message("",text)
Fileput(file,text)
If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
map = MapFileGetCSV(file)
offset =  Int(MapKeyFind(map, "ii" , '0'))
text="u="
uVal = MapKeyFind(map, "u" , '')
For i = 1 To Strlen(uVal)
   char= StrSub(uVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next


text=text:@CRLF:"p="




pVal = MapKeyFind(map, "p" , '')
For i = 1 To Strlen(pVal)
   char= StrSub(pVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next
text=text:@CRLF:"f="
fVal = MapKeyFind(map, "f" , '')
For i = 1 To Strlen(fVal)
   char= StrSub(fVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next
Message("Decrypted",text)







NOTE: when can be expect to see double quotes shown in code blocks....


NOTE_1: I messed that up... meant single quotes instead of #39

[td]

NOTE_1: I messed that up... meant single quotes instead of #39

Not any time in the near future. You could do what I plan to do and just write a script to convert before you post or I may just go back to my old BBC converter script that allows me to embed scripts directly in the text.

[stanl]

corrected

Code (WINBATCH) Select Expand



file = "C:\temp\testhash.sys"


If fileexist(file) Then filedelete(file)
offset = Random((255 -122)/2)
text = "i,v":@CRLF
text = text:"ii,":offset:@CRLF:"u,"
var = "SmileyDan"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
text = text:@CRLF:"p,"
var = "CuriousMelba2$"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
text = text:@CRLF:"f,"
var = "c:\temp\pf.sys"
For i = 1 To Strlen(var)
   char= StrSub(var,i,1)
   text= text:num2char(char2num(char)+offset)
Next
Message("",text)
Fileput(file,text)


If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
map = MapFileGetCSV(file)
offset =  Int(MapKeyFind(map, "ii" , "0"))
text="u="
uVal = MapKeyFind(map, "u" , "")
For i = 1 To Strlen(uVal)
   char= StrSub(uVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next
text=text:@CRLF:"p="
pVal = MapKeyFind(map, "p" , "")
For i = 1 To Strlen(pVal)
   char= StrSub(pVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next
text=text:@CRLF:"f="
fVal = MapKeyFind(map, "f" , "")
For i = 1 To Strlen(fVal)
   char= StrSub(fVal,i,1)
   text= text:num2char(char2num(char)-offset)
Next
Message("Decrypted",text)




[stanl]

Either what follows will make some sense, or I make a complete ass of myself: :o

   
• I can create the testhash.sys file with either WB or PS.
• In WB it can be decrypted as Map values, in PS as Hash values
• WB cannot convert the file to Map if created by PS. PS writes to UTF16 by default so null values; and if I explicitly set -Encoding utf8 the file still contains BOM.
• PS cannot read the file as a hash if created by WB.

[EDIT]   PS version 6.1 does have an encoding as utf8NoBom....

Maybe time to see if WB and PS can talk to each other as arrays, even if they don't need to.

[td]

Almost all WinBatch file functions handle UTF-8 and UTF-16 files with or without a BOM. The exceptions are some CSV-related functions.

[stanl]

OK. Just for fun... beat the dead horse. I have attached 2 text files, one created with PS, one with my WB script. In both cases I hard-coded the offset to 80 [the second line in the files]. Pull them both up in Notepad++ and WB encoding is ANSI, PS is utf8 with BOM. Neither can read the others file to decrypt. This actually worked to my advantage as I had to pass muster with some basic encryption and so far have foiled attempts of testers to decrypt.... 


Below is the simple PS code to test encrypt/decrypt with lines commented for testing the attached files.

Code Select Expand



function set-rot {
[cmdletbinding()]
Param ([String]$text,[Int]$offset)
$rot=""
foreach ($char in [char[]]$text) {
   $n = [Int]$char + $offset
   $newchar = [char]$n
   $rot += $newchar
   }
Return $rot
}
function get-rot {
[cmdletbinding()]
Param ([String]$text,[Int]$offset)


$rot=""


foreach ($char in [char[]]$text) {
   $n = [Int]$char - $offset
   $newchar = [char]$n
   $rot += $newchar
   }
Return $rot
}
$file = "C:\temp\testwb.txt"  #for testing
[Int]$offset = Get-Random -maximum 255 -minimum 122
$offset = $offset / 2
$str = "i,v" + "`r`n"
$str += "ii," + $offset +  "`r`n"
$var =  set-rot -text "SmileyDan" -offset $offset
$str += "u," + $var +  "`r`n"
$var = set-rot -text "CuriousMelba2$" -offset $offset
$str += "p," + $var +  "`r`n"
$var = set-rot -text "c:\temp\pf.sys" -offset $offset
$str += "f," + $var


#uncomment next 2 lines to create file
#$str
#$str | Out-File $file -Force -Encoding utf8


$Hash = Import-Csv $file -Encoding UTF8 | Group i -AsHashTable
$user = get-rot -text $Hash.u.v -offset $offset
$pw =  get-rot -text $Hash.p.v -offset $offset
$pp =  get-rot -text $Hash.f.v -offset $offset
$user
$pw
$pp


[td]

Not sure what you are trying to make - I likely missed something - but you can write and read UTF-8 files in WinBatch. WinBatch will automagically convert the Unicode and add a BOM for  UTF-16 but you have to indicate the conversion type using the ChrSetCodePage function and add the BOM yourself for UTF-8.

[stanl]

I did play around with ChrSetCodePage and there is a way to make PS output utf8 w/out BOM. I was just surprised that while there is PS encoding for Ascii, there isn't one for Ansi...  No harm/no foul. If I want PS to decrypt data written from WB [using a random number ROT] I'll write the WB script to call PS code via the CLR. 

[td]

Forgive the typos in my previous post. Spent a long day doing some technical writing...

[stanl]

NP. Didn't think typos were ever brought up. We can conclude with War Games (1983)... [edited] "the only way to win the game is not to play"

[td]

No typos were not mentioned by anyone other than yours truly. Just happened to notice my typed nonsense in passing and thought it merited an apologetic mention of some kind.

[stanl]

I made the original post because of the difficulties I had with WB code to read/decrypt a comma-delimited text file created by PS - that file using a random number offset to encrypt the second section of the file. To decrypt the file with PS I could use their Import-Csv cmdlet piped into a hash table.  By default the PS code would output the encrypted file as utf-16 LE BOM, but I also could change the output coding to utf8 with or without BOM. 


Trying to read the file with WB code either as a Map [from csv] or Array [from csv] failed. I added ChrSetCodepage(65001) for utf16 and played with other combinations but kept getting WB errors that file contained null chars. 


On the other hand, if I created the output file with WB, it could be encrypted/decrypted and the encoding was ANSI - which PS couldn't decrypt with Import-Csv. 


There is a likelihood I am doing something wrong with WB code to decrypt PS output, or I will have to choose an alternate to Import-csv on PS side to decrypt WB output.


Or - Apples and Oranges - no need for the 2 apps to communicate with each other in this instance. Besides, isn't "quiz for duck" a Firesign Theatre reference?

[td]

If by reading to an array you mean the ArrayFileGetCsv function in combination ChrSetCodePage, this will, as you mentioned, definitely not work. WinBatch CSV file functions simply do not support UTF-16 or UTF-8. As for why that is the case? I have no idea. It could have been an oversite or some kind of philosophical design decision or something else entirely.

The reason I mentioned ChrSetCodePage is that it is possible to read a UTF-8 file using other WinBatch file functions by using ChrSetCodePage to convert from UTF-8 to UTF-16 on the fly. It is also possible to read a UTF-8 file into a binary buffer and then convert the extracted content to UTF-16. I brought this up originally as an aside and not as a solution suggestion.

These ad-hoc discussions are useful in that they provide use-case examples. Knowing what users are up to is also a tremendous help in future design and sometimes leads to the discovery of defects (bugs) or limitations that need correction.

For what it is worth, when I use the term ANSI I am referring to Microsoft's Windows 125x series code pages. For many western languages including US English, that series is identical to ANSI. 

[stanl]

Yeah... we can end on a good note. I finally dug up a PS function to detect encoding. It determines the rot file created by WB is Ascii but using Import-Csv to make a hash table fails unless the file is utf8 or unicode. Some more stuff to play around with but won't bother this thread anymore ;)

[stanl]

One last shot... maybe use Json Extender to decrypt if file output as Json fom PS


[EDIT]... haven't worked with WB Json extender for a year or more... seems to say file is Json... but that's about it.

Code (WINBATCH) Select Expand



file = "C:\temp\testhash.txt"
If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
json = FileGet(file)
Valid = jsValid(json)


If !Valid
Message("Not Valid Json",file)


Else
  Object = jsParse(json)
  TypesMap = jsConMap(Object, @JsonType)
  Found = @false
  ForEach Key In TypesMap
     If TypesMap[Key] == @JsonStr
        Found = @true
        Break
     EndIf
  Next
  ; Found a nested JSON object?
  If Found
     Path = jsKeyPaths(Object, Key)
     Text = jsValueGet(Object, Path[0])


  Else
     Text = 'No nested objects'
  Endif
  Message('Nested Object Search', Text)
Endif
jsConClose()
Exit


[td]

It would be best if you remembered that WIL maps must have unique key names and can't hold duplicate keys. The map returned contains a numeric key name (since it is an array) and a handle for each JSON object in the array. Those objects each have two keys of there own.  Since your goal is quick lookup, it doesn't make a lot of sense to blindly convert the array to a map.

[td]

Don't think this is particularly useful.  Just curious to see what happened with all the conversions since WIL maps only support ANSI keys. They do support any valid WIL datatype in values. Except for arrays of course.

Code (winbatch) Select Expand

file = "C:\temp\testhash.txt"
If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
AddExtender("ilcjs44i.dll", 0, "ilcjs64i.dll")
json = FileGetW(file)
Valid = jsValid(json)

If !Valid
   Message("Not Valid Json",file)

Else
   Object = jsParse(json)
   ValueMap = jsConMap(Object, @JsonValue)
   Found = @false
   ForEach Key In ValueMap
      Elem = ValueMap[Key]
      Whatever[jsValueGet(Elem,"v")] = jsValueGet(Elem,"i")
      ; Could call jsConClose on each "Elem" here to conserve  resources
      ; but in most cases, it is not necessary.
   Next
Endif


jsConClose()

;; Check the map keys.
Message("Keys", MapKeysGet(Whatever, 2, @lf))   

Exit

[stanl]

If I use Out-File in PS to save the data as Json it is encoded UTF8-BOM; if I use Set-Content it is Ansi. If the former the WB code will indicate not valid Json but will decrypt in PS. If the latter, the code you posted will display the data, but PS will error if trying to decrypt. Good to keep in one's back pocket.

[stanl]

If I use Out-File in PS to save the data as Json it is encoded UTF8-BOM; if I use Set-Content it is Ansi. If the former the WB code will indicate not valid Json but will decrypt in PS. If the latter, the code you posted will display the data, but PS will error if trying to decrypt. Good to keep in one's back pocket.

This gets close. I attached an updated testhash.txt file in Ansi encoding. I then hacked up the code to attempt a decrypt after finding the offset value in the Json.  User name decrypted, the password and passphrase were 1 char off. In looking again at the Json Extender, there is a serialize function that permits encoding. So maybe WB can create Json that PS can read and decrypt.

Code (WINBATCH) Select Expand



file = "C:\temp\testhash.txt"
If ! fileexist(file) Then Terminate(@TRUE,"File Not Found",file)
AddExtender("ilcjs44i.dll", 0, "ilcjs64i.dll")
json = FileGetW(file)
Valid = jsValid(json)


If !Valid
   Message("Not Valid Json",file)


Else
   Object = jsParse(json)
   ValueMap = jsConMap(Object, @JsonValue)
   Found = @false
   decrypt = ""
   offset = 0
   ForEach Key In ValueMap
      Elem = ValueMap[Key]
      var = jsValueGet(Elem,"i")
      If var == "ii" Then offset = jsValueGet(Elem,"v")   
   Next
   Message("Offset",offset)
   If offset<>0


      ForEach Key In ValueMap
         Elem = ValueMap[Key]
         var = jsValueGet(Elem,"i")
         If var <> "ii"
            text = var:"-"
            var1 =  jsValueGet(Elem,"v")
            For i = 1 To Strlen(var1)
               char= StrSub(var1,i,1)
               text= text:num2char(char2num(char)-offset)
            Next
            decrypt = decrypt:text:@CRLF
         Endif
      Next
      Message("Values",decrypt)
   Endif
   jsConClose()
Endif
Exit