d = random,randint(1,30)
data = [d, strftime("%Y%m%d %H%M%S", gmtime())] #random num , system time
client.publish("gas", str(data)]
This is a part of my python code which is ver2.
I'm trying to send a list using MQTT.
However, If I write bytearray instead of str which is third line
It says "ValueError: string must be of size 1".
So I wrote str then make it sting type
Can I send a just list which is NOT string type.
MQTT message payloads are just byte arrays, there is no inherent format to them. Strings tend to works as long as both ends of the transaction are using the same character encoding.
If you want to send structured data (such as the ost) then you need to decide on a way to encode that structure so the code receiving the message will know how to reconstruct it.
The current usual solution to this problem is to encode structures are JSON, but XML or something like protobuffers are also good candidates.
The following question has some examples of converting Python lists to JSON objects
Serializing list to JSON
Related
This question has been stumping me for quite a while now. I have a binary file, within which are the contents of an SNMP trap sent by a server, i am able to read the file and output its data as a UTF-8 encoded string '0‚Œcommunity¤‚} +…"Õ #À¨V ÁC8o[0‚W0+…"Õ CPU00020A+…"Õ /Message Detailing The Event Triggering The SNMP Trap (BIST).0+…"Õ 0+…"Õ 7K882L30+…"Õ server-name0+…"Õ 0+…"Õ N/A0+…"Õ "1"0+…"Õ 7K882L30%+…"Õ Main System Chassis0&+…"Õ iDRAC-server.example.com' The issue is, im trying to extract certain elements of this data such as community and server-name and store them into variables, however, the program needs to work with lots of different SNMP traps with values that may differ in length and content, and as I don't know where the data will be located within the string I cant come up with code to reliably sort the data into the specific variables. The only thing I have to go off is that I know the sequential order of the data.
I'm not sure where to begin with approaching this problem, many thanks for any help.
To clarify, the data is currently stored in a variable char buffer[1025] = DATA;
How to pass map object in AWS SQS queue?
I have a map object coming with some data and I wanted to pass the same map
in AWS SQS queue in order to persist that. How can I do the same?
I imagine you want to store the map so you can digest it from SQS later, as such I recommend you serialize it. You should serialize your data, as opposed to just sending the map into SQS and trying to parse it back into a map later, as it is much safer and likely easier.
JSON and MSGpack are popular for serialization and deserialization.
Here's some pseudo code:
some_map = {name: 'tom', age: 31}
serialized_map = make_into_json_function(some_map)
sqs.send_msg(queue, credentials, serialized_map)
To turn the JSON back into a map:
message = sqs.get_msg(some_msg_id)
new_map = make_json_into_map_function(message.text)
print new_map
Here is what I did (and it is working 100%):
Tried to send map values as attributes in SQS : using setMessageAttributes, but since in SQS you can send max 10 attributes at a time thus it was not suiting my requirement (I was having more than 10 values in my map).
SQS provides an option to send message using messageBody as well so I converted my map into plain Json string and sending it to SQS.
Thanks!
I think you need to serialize your map into a XML, JSON, or unformatted text.
When you receive messages, deserialize it to the map.
JSON can be a good choice, and you can use jackson or gson to serialize/deserialize JSON.
Here are two SQS limitations you should keep in mind:
Message content:
A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed: #x9 | #xA | #xD | #x20 to #xD7FF | #xE000 to #xFFFD | #x10000 to #x10FFFF
Any characters not included in this list are rejected. For more information, see the W3C specification for characters.
Message size:
The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB).
I am totally new to Python. I have to parse a .txt file that contains network byte order binary encoded numbers (see here for the details on the data). I know that I have to use the package struct.unpack in Python. My questions are the following:
(1) Since I don't really understand how the function struct.unpack works, is it straight forward to parse the data? By that, I mean that if you look at the data structure it seems that I have to write a code for each type of messages. But if I look online for the documentation on struct.unpack it seems more straight forward but I am not sure how to write the code. A short sample would be appreciated.
(2) What's the best practice once I parse the data? I would like to save the parsed file in order to avoid parsing the file each time I need to make a query. In what format should I keep the parsed file that would be the most efficient?
This should be relatively straight forward. I can't comment on how you're actually supposed to get the byte encoded packets of information, but I can help you parse them.
First, here's a list of some of the packet types you'll be dealing with that I gathered from section 4 of the documentation:
TimeStamp
System Event Message
Stock Related Messages
Stock Directory
Stock Trading Action
Reg SHO Short Sale Price Test Restricted Indicator
Market Participant Position
Add Order Message
This continues on. But as an example, let's see how to decode one or two of these:
System Event Message
A System Event Message packet has 3 portions, which is 6 bytes long:
A Message Type, which starts at byte 0, is 1 byte long, with a Value of S (a Single Character)
A TimeStamp, which starts at byte 1, is 4 bytes long, and should be interpreted an in Integer.
An Event Code, which starts at byte 5, is 1 byte long and is a String (Alpha).
Looking up each type in the struct.unpack code table, we'll need to build a string to represent this sequence. First, we have a Character, then a 4Byte Unsigned Integer, then another Character. This corresponds to the encoding and decoding string of "cIc".
*NOTE: The unsigned portion of the Integer is documented in Section 3: Data Types of their documentation
Construct a fake packet
This could probably be done better, but it's functional:
>>> from datetime import datetime
>>> import time
>>> data = struct.pack('cIc', 'S', int(time.mktime(datetime.now().timetuple())), 'O')
>>> print repr(data) # What does the bytestring look like?
'S\x00\x00\x00\xa6n\x8dRO' # Yep, that's bytes alright!
Unpack the data
In this example, we'll use the fake packet above, but in the real world we'd use a real data response:
>>> response_tuple = struct.unpack('cIc', data)
>>> print(repr(response_tuple))
('S', 1385000614, 'O')
In this case, the 3rd item in the tuple (the 'O') is a key, to be looked up in another table called System Event Codes - Daily and System Event Codes - As Needed.
If you need additional examples, feel free to ask, but that's the jist of it.
Recommendations on how to store this data. Well, I suppose that depends on what you'd like to do long term to this data. Probably, a database makes sense here. However, without further information, I cannot say.
Hope that helps!
I am programming a CGI script using c++ where I get post-data from client
If I have post-data like this
sam=1&sam2=3&sam5=65
what are the steps to parse options where I will put each pair in map object in c++?
I have thought of splitting values by '&' into vector array then splitting each entry in the vector array by '=' and put them in map object, what do you think? Also I want to know what can I do to deal with malformed post-data.
You have the first two steps correct. Separate the name/value pairs at the & and separate the name from the value at the =. But then you have to URL decode the name and URL decode the value before inserting them into the map.
For example, a + in the name or the value should be changed to a space. A %2B should be replaced with a space. A %3D with an equals sign.
As for malformed post data, it will really depend on your application what you consider malformed. You probably should check for an ambiguous value like x=2=3. Is the value 2=3 or 3? And you can check for a missing name, like =3 or just 3.
But the rest is up to the specifics of what's legal in your particular application.
I have a candidate key (mongodb candidate key, __id) thats looks like the following in protocol buffers :
message qrs_signature
{
required uint32 region_id = 1;
repeated fixed32 urls = 2;
};
Naturally I can't use a protocol buffers encoded string (via ParseToString(std::string)) in my bson document since it can contain non-printing characters. Therefore, I am using the ascii85 encoding to encode the data (using this library). I have two questions.
Is b85 encoding bson-safe.
What is bson's binary type for ? is there some way that I can implant my (binary) string into that field using a mongodb API call , or is it just syntactic sugar to denote a value-type that needs to be processed in some form (--i.e., not a native mongodb entity)?
edit
The append binary api's show's data being encoded as hex(OMG!), base85 is therefore more space efficient (22 bytes per record in my case).
BSON safe, yes. The output of ASCII85 encoding is also valid utf-8 iirc.
It's used to store chunks of binary data. Binary data is an officially supported type and you should be able to push binary values to BSON fields using the appropriate driver code, BSONObj in your case. Refer to your driver docs or the source code for details.