Which real-time communication protocol between hardware and web components?

I wasn't entirely sure how to word my question in the title so apologies if it's confusing.

I'd like to build a system that would function as a sort of information dashboard for my home. It would consist of a number of hardware and software components that would ultimately result in a simple, clean website with real-time displays of a number of analog sensors such as temperature, wind speed and direction, etc.

I've got a good idea of what I'm going to do for the hardware, as well as for displaying the information; my question has to do with the communication between the hardware and web server.

I'd like the hardware to fire messages at a fairly fast rate so I don't think HTTP POST will suffice. I'm also not extremely concerned with receiving 100% of the messages but receiving as many as possible is definitely a plus. The data will be coming from the hardware, populating some sort of database (likely Redis).

So far, I've researched a couple of things but I'm not sure I'm heading in the right direction. I've looked in to message-oriented middleware such as RabbitMQ but I'm not convinced I need the overhead. I've also looked into Redis Pub/Sub which seems like a more appropriate solution since I'd like the web app to chart out say the last 5 minutes of data but even then I'm not certain. Can I just fire UDP packets to a custom-built listener?

I'm pretty certain the hardware will be two stages (a uC feeding a small embedded linux machine)开发者_开发技巧 so you might even liken this to desktop software firing messages to a web server as quickly as possible.

I'm venturing into an area that I know absolutely nothing about so any guidance is much appreciated.

To communicate between your data acquirers and collectors, you might consider the industry standard Modbus TCP protocol. (In a previous life, I wrote network code for programmable controllers.)

I'm sure there are libraries available for most microcontrollers, though they might not be open-source, but I doubt a JS version of Modbus exists so you'd need to write the server-side lib yourself. As I recall Modbus isn't particularly complex, especially if you don't use some its more esoteric features. Of course, writing this got me thinking how I'd write such a thing and lo and behold, it's already been done for nodejs! (One of the many reasons I love the nodejs community!)

So thats the easy answer...now, with my hacker-hat firmly in place...

You mention that your HW will feed one or more "small embedded Linux machine(s)".

Have you considered running nodejs on each data collector? If the size of nodejs' executable is the issue, I'm sure there are large parts of its out-of-the-box functionality that could be removed or moved into modules.

I realize what I'm recommending is not a small undertaking--porting an application the size and complexity of nodejs/V8 to a new platform is certainly challenging--but I strongly suspect nodejs' event-driven design would be an excellent match for data acquisition, discrete manufacturing, process control, and other manufacturing applications.

Like the other poster mentioned, your're not going to have issues with http post. Node's http implementation is built for high concurrency.

Personally, I think I'd go with:

1. Hardware device output ->
2. Linux box fires an http post directly to your central server (node.js) ->
3. Take your changes and immediately publish them to your web client via socket.io (real time transport for the browser). https://github.com/LearnBoost/Socket.IO/

Socket.io is probably the best out of the box real time transport for node.js <==> browser

If you want persistence, redis isn't bad (plus you get the free pub/sub) if your data fits into that model, which it probably does.

There's no reason you couldn't use a reglar tcp based connection also (net module) if thats your bag. Unless you want the data to be unreliable, I wouldn't go to udp. Thats more lossy streaming media oriented.

I really doubt that you're going to have enough messages to worry about a dedicated message queue. Rabbitmq introduces features like queue persistence and is built for incredibly high throughput. Probably orders of magnitude more than what you're after.

You might look at a library like zeromq for which there are node bindings: https://github.com/JustinTulloss/zeromq.node. Its like a topic/other type of exchange in rabbitmq, but without a central message queue node (one calls this brokerless). That way you can just talk directly to your linux/hardware nodes, but still get the message queue like interface -- you publish your hardware updates on a 'channel' and your server nodes listens for such updates.

What's wrong with http post? If you are using node.js as your web server it should be plenty fast enough. You are already coding the presentation layer in node so that's one less tool you will have to use, and in this case it's a perfect fit. Keep it simple and stick with node.