fix(vm): boolean table keys, nan sentinel, set! encode errors, # on non-tables

[dimamik]

Thanks a lot for building this! We're about to release Legion, one of the sandboxes of which will be Lua.ex, and we stumbled upon a few non-urgent thingies around data transformation that we think are worth looking into:

• Value.encode/3 turned true/false map keys into strings, so %{true => 1} came back as {["true"] = 1}.
• The :nan sentinel was not a number at the boundary: type(0/0) was "userdata", math.type(0/0) was nil, tostring(0/0) was ":nan", and Lua.decode!/2 raised on it.
• Lua.set!/3 crashed with a bare FunctionClauseError on values Lua.encode!/2 refuses with Failed to encode.
• # returned 0 for nil, booleans, numbers and functions instead of raising. Fixes length operator on nil/boolean returns 0 instead of raising #404.

Thanks in advance, and let me know what you think!