I thought I ought to do a short update of how things move forward.   Well, despite me working all time I possibly can find with the new version of VSCP & Friends I still just move very slowly forward. On the positive side is of course that I write “forward” and not “backward”, but I feel things are moving very slowly ahead right now.  But it is just a lot to do and I am only one person. You bet I have wet dreams about having a team of programmers working on this.

Tables

My current work is on the new “tables” that will use a database. Reachable from all interfaces (tcp/ip, mqtt, rest, websockets) they allow for a way to collect and fetch series of measurements as the old ones did. Different this time is that Sqlite3 is used as the storage engine and that they are a lot more flexible than they where before.

The goal with tables is to be able to drag a decision matrix element in place that collect the data form a device. You have to do that manually today. And on the other end drag a live diagram/table to a web page that display the content in the table.

Anyway if all goes well I hope the full table functionality is fully in place in two weeks or so.

Discovery

Discovery is the next task. Today one can listen on the multi-cast channel for new devices that are discovered. This will be developed further so that more information about a node is published. This means that with discovery the VSCP daemon itself will go out and fetch data from nodes that it discovers. The VSCP daemon will also download and cache MDF’s.

Also new is that discovery data will be stored in a database and this data can be fetched and investigated through the different interfaces (tcp/ip, mqtt, rest, websockets).  In this database it will be possible to assign a real text name to a node and this name will in the future be possible to use when one refer to a node instead of using it’s GUID.

Lastly I hope to make a push interface to this discovery mechanism so that a VSCP aware phone for example can start an app. when a new device is discovered.

Hopefully in place before the summer. But don’t hold your breath.

Next release

The reason why I hold the next release of VSCP & Friends is that I the database structures aren’t final yet, and they will not be final until the interfacing code is in place. That is mainly the web interface code as of today, but also VSCP Works. If I release before the database structures is final I have to write update code between releases and that takes time I don’t have. So this work has to take the time it takes.

For those of you that want to try the head code, please do. But be careful to delete database files when you rebuild the project (rm /srv/vscp/*sql3).  The VSCP UX code now have a session window and variable handling and a few other functions in place and may be worth a try.

Community

It saddens me of course that VSCP has not been successful in the forming of a development community.  Today we don’t even have an active test or discussion community. This is of course harder still from a developer perspective as endless hours of work from my end just appears to end up in /dev/null.  But I still strongly feel that I will try to make the vision of a uniting system set up for soon seventeen years ago come to reality  before I give this up.  This year has been the hardest yet on this journey. and without all your kind donations I would not have been able to proceed at all. Thanks!

If someone out there share this passion or a tiny little tiny bit  of it, take on some VSCP work.  Web ux interface and bindings to different languages is probably the most important tasks at hand.  But there are so much more.

Have fun!
/Ake

Sync

To ask for readings (measurements/sensor values) from nodes one can use CLASS1.CONTROL, Sync. One can use sensor index, zone, subzone as a way to tell which readings are of interest. A node should send the requested value immediately when it get a sync. For cashing, use tables or variables to store the readings.

The sync’s primary use is to synchronize sensor readings from multiple nodes in time but it can obviously also be used  just to request data.

You can see that modules from Grodans Paradis AB implement the sync but also have registers the data can be read from and automatically repetitive delivery of sensor data.

For the Kelvin 1-Wire module for example the sync has the following description

If a SYNC event is received by the module it will check the zone/subzone parameters of the event and send out temperature measurement event(s) for all sensors that match. This can be a handy feature to use of one want synchronized data from several sources.

But you can also instruct the module to output temperature sensor data with a set time interval (see report interval registers here) or read the temperature reading in a register (see temperature registers here).

Periodical events are perfect when the VSCP daemon is part of a system. Tables and remote variables is two features it have that is well suited for this type of sensor data.

Tables is being rewritten and extended from the functionality available in the last release at the moment and is mostly into place in the head code right now. There is a description here. One can do many things with sensor data in them and actually create user defined SQL databases if one like that which can have its data requested over REST/TCP/IP/websockets…

If one want the x last readings the static table is perfect as it works in a round robin fashion.

One can also use remote variables to store certain sensor data. In many of my setups I use them all the time. A typical sample is a sensor value that should be sent on specific intervals to a cloud service but where internal systems need this sensor value at a much higher rate than the cloud service need it. The easiest way to do this is to store the value in a variable with a DM entry when it is received. This way one have the sensor value accessible all the time and  request the latest value over any of the interfaces. To send the value to a cloud service one use one of the internal events for example the minute event and send the value there calling an external script or using one of the internal actions such as the http put/get/post action or even the run internal JavaScript action (soon LUA to).

Well that’s power for you!

GUID’s

A GUID is the one and only id for all VSCP devices, It is globally unique, 16-byte, and will never change. It can be derived from some common id’s like MAC addresses or come from a user/company assigned “namespace” which can be requested for free.

At the application level this is the id “that should” be used if one want to have an id that absolutely does not change over the life of a device.

The problem with 16-byte id’s is obvious and specially clear on CAN devices. There is no room for it in the payload. One see this in wireless payloads also. VSCP therefore uses “nicknames” for devices to identify a unit on a local segment/bus. A nickname is a number (any) that directly translate to a GUID. For CAN this nickname is a 8-bit number that is not zero (reserved for a possible master) and not 0xff (reserved for an uninitialized node).

On CAN a nickname can be assigned using the automated nickname discovery or be hard assigned. In the case of nickname discovery a possible master is search first and if there this device is supposed to assign a node id to the node. This is NOT implemented in the VSCP daemon (but may be) and must be handled in user land. That is one listen for CLASS1.PROTOCOL, New node online/probe to node zero and respond to it if it is detected and after that assign a nickname.

The nickname will normally not change on a bus. And one can make sure it does not by assigning the old nickname to a node that for example is replaced. But it can not be guaranteed so it is better to use zone/subzone’s as a way to connect “groups” together. That way when a node is replaced one only need to load it’s registers with the original nodes content and it is a replacement that is a real copy of the old device.

So at this point one have a one to one mapping between the GUID of the node stored in standard registers 0xD0-0xDF an the nickname. A table than must be added to/changed when a CLASS1.PROTOCOL, New node online/probe comes in by reading the GUID of that node.

One can easily do a REST i/f <-> CAN4VSCP interface app here if one want that. For example is a uvscpd planed that only have a driver interface and a link to a tcp/ip interface on a remote VSCP daemon.

Back to the uniqueness of the GUID. The VSCP daemon confuses things. This is because the fact that VSCP actually only identify things with VSCP GUID’s. So a Level I driver that implements the CANAL interface will receive and send using nicknames on the lower part but will use gull GUID’s on the higher part. This means that a device with GUID-A pass an interface (VSCP Level I in this case) and the will look as if it comes from GUID-B. Well one also send events to GUID-B.

The thing here s that the interface also is identified by a GUID. Either server assigned or user assigned. This is true for all interfaces on the VSCP daemon. It is important to note that this GUID can change when a daemon is restarted because the interface becomes another one.

Interface GUID’s of the VSCP daemon is built like this

XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:aa:aa:bb:bb

0xaaaa form a server assigned id for the interface. 0xbbbb form the nickname and is actually 00bb for a Level I driver where bb is the nickname.

So for a user sitting on the REST/TCP/IP/websocket/mqtt interface a node that has a GUID YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY may look like it is XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:aa:aa:bb:bb. The only way to know the real GUID of a node is to read its GUID with CLASS1.PROTOCOL, Extended read register for example.

Again it is a one to one relationship and a CLASS1.PROTOCOL, New Node online from that interface may changed things so the table needs to be updated.

Not hard to implement on the application level.

That said the next version of VSCP & Friends will have a new function called node discovery. This is partly in place today also as new nodes are announced on the VSCP multicast channel. But this will be extended so nodes can have a unique name instead of the GUID and also the daemon will keep an internal table of “real GUID’s” one can query. Also MDF’s will be cashed by the server.  This is what I will take on next after the new tables functionality I work on now is finished. Hopefully released before the summer.

In the meanwhile if one assign a GUID to a driver. And make sure that nodes use the same id, that is is initialized only once, the interface GUID is fine as a node identifier if one skip the 0xaaaa interface part of

XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:aa:aa:bb:bb

and just compare the rest. Also the one to one relationship works then to the “real GUID”.

VSCP and unique id’s.

A node’s assigned nickname id will be retained after a reboot on a CAN4VSCP bus device (or should). It is only changed (or can be) if the configure buttons is pressed. The first free id will be used in the nickname discovery process. In practice I see that this always turn out to be the same id in a real system but if one has removed other nodes in between one can never be certain. But it still is not a big problem for a service person. He/she in the first place press the configure button so it should not be a big problem to check that the id turned out to be the same and manually change it if not. The only time the configuration is done is during install and replacement of a node anyway. Another schema than the nickname discovery is to preferred for nodes that are going in and out on a CAV4VSCP bus.

The single most important part of a VSCP system is the Register Abstraction Model.  With it you can describe anything that are real (or even not real) in this universe.  Quite a piece of a powerful super tool to have in your arsenal.

In the IoT world people often say that MQTT,  CoAP,  etc is IoT servers/protocols. For me they are transport mechanisms. Good ones, at least in the case of MQTT.  Useful for IoT? Yes. But IoT severs no?

Now what!?

To explain my point. Look at a web server. It serve content using the HTTP protocol. Simple enough. Works good. Quite similar to MQTT in fact.

But…

this was not the reason the world-wide web was a success.  If everyone started to serve different non standardized content with the http protocol it would have been a mess of it all. Very hard to cope with for the web readers at least.  Yes, it would have been useless. Well this was more or less the case also pre HTML.  But what HTML did was to specify a common format for the content. That was the killer that made everything moving. Not the web server which is just the transport mechanism just as MQTT and others are in IoT.

VSCP try to be more than a transport mechanism for m2m and IoT. A common way to look at and handle “things”.  (yes MQTT can transport it just as CoAP and others can)

In the picture above is a view on how VSCP look at a “thing“.  A “thing“can be a TV,  a refrigerator,  a car,  a lamp,  a human or whatever. Everything can be described in this way. Yes, everything. Hardware as well as software or a some abstract non existing life form. ANYTHING is the world.

To describe anything is of course not a problem. One just describe the anything in a way that comes up in one’s mind. So if we ask 999 people to describe the anything object there will be 999 different descriptions. That is actually OK. The problem is rather that there will be 999 different ways to interpret the descriptions of the anything. Hard work.

If we instead came up with a common way to look at the anything,  an abstraction of it,  we could all use that abstraction to describe every the anything and after that we do not need to come up with so many new ways to interpret the descriptions nor ways to interact with the anythings of the world. We could even use this abstraction on top of any of the 999 description from above as it is useful for describing anything. Good if we love our “description” and don’t want to throw away our lovely description and interpretation of it.

Of course, the abstraction would be most useful if it was available for a high-end system or user as well as for low-end system or user.  Low end can actually be really  low-end.  So we better start there.  But a high-end user or system demands handling of great complex things so we need to take this into account as well.

VSCP solve this in the following way

• Everything can be described by a collection of  8-bit registers.
• To know whats inside an 8-bit register you need the ability to read it.
  
• To change the content of an 8-bit register you need the ability to write it.

So VSCP can describe everything in the universe,  yes everything from a flower,  a person,  a car or anything else with a set of 8-bit registers. The only thing needed to interact with the resister set is that registers can be read and written.

So

• Read a byte.
• Write a byte.

is all that is needed for  VSCP to work with “anything” on its lowest level.

In VSCP this is called the Register Abstraction Model.

VSCP’s register abstraction model demands that all “anything” objects have a set of registers that contain specific information. Same on all. The two most important parts is GUID, ett globally unique id  a 16 byte world unique if for “the anything“,  and a 32-byte area which contains a pointer to a  Module Description File.  The GUID identifies “the anything” much the same way as an Ethernet MAC does for Ethernet device.  They can actually be coupled. The MDF pointer can point at a local location or to a file on the Internet. The content is a well-defined XML description of “the anything”.

One thing that is described is of course the registers of “the anything“.  But to handle things on a register level can be to hard sometimes. For instance is a floating point value stored in eight registers. If this floating point value for instance represent a temperature of “the anything”  and this is what the application or user is interested in there should be no need to know how it is actually stored on “the anything“.

So to make life easier for high-end systems or users the MDF often also contains abstractions. Abstractions specify higher end type such as strings, floats, integers, booleans and tell in what registers they are stored and also how many bytes they use.  So a high en user just handle these higher end objects, reading and writing them,  even if the system still only read and write bytes.

Both registers and abstractions describe the same “anything“,  just on different levels.

At this point it would be possible to build an application that can configure any device, “anything“,  just by read/write byte operations and by reading and interpreting the MDF of the unit. The user (or machine) just fill in the values. The same application can be used to configure a car,  a TV,  a clock radio etc.

The MDF can also contain wizards. A wizard can guide a human user, step by step through a specific setup of a device. They are also described in an abstract way and can be interpreted by C/C++, JavaScript and other higher end programing languages alike.  So it is as easy to make an app. that goes through the wizard steps as it is to make an HTML page that do the same.

It is of course possible to use XML/JSON on the higher level to “set register/abstractions”.  A read and a write byte operation is still all that is needed.

The fact is that the register abstraction model is so light weight that it easily can live alongside a more proprietary solution

But way?

That you have to think about yourself… ,-)

ps VSCP is of course a lot more than this. There are events, classes/types, bootloaders, decision matrices and a lot of other clever things. ds.

I will probably starve to death doing this but  the following hardware modules will be open source again.

can4vscp_accra – Counter module.
can4vscp_sanfrancisco_barometer  – Barometer module.
can4vscp_humidity – humidity and temperature
can4vscp_beijing – I/O- module.
can4vscp_vilnius – AD- module.
can4vscp_odessa – Expansion module.
can4vscp_kelvin_1wire – 1-wire temperature module.
can4vscp_frankfurt_rs232 – RS-232 interface to CAN4VSCP
can4vscp_frankfurt_gprs – GPRS interface to CAN4VSCÅ
can4vscp_newyork_lcd_display_controller – Display controller
can4vscp_frankfurt_eth – Ethernet <->CAN4VSCP
zeus_expansion_roma – Ethernet controller.
zeus_expansion_moscow
can4vscp_power_injector
zeus
zeus_expansion_napoli
zeus_expansion_leicester
zeus_expansion_kiev
zeus_expansion_budapest
zeus_expansion_brussels
blitz
thunder
can4vscp_frankfurt_ble
can4vscp_frankfurt_bridge CAN4VSCP <-> CAN4VSCP
can4vscp_frankfurt_rs485 RS485 <-> CAN4VSCP
can4vscp_frankfurt_usb  USB <-> CAN4VSCP
can4vscp_kelvin_9700
can4vscp_kelvin_multisensor
can4vscp_kelvin_smt160
can4vscp_kelvin_thermocouple
can4vscp_Loos_temp_control

The following hardware modules will be closed source from today

can4vscp_accra – Counter module.
can4vscp_sanfrancisco_barometer  – Barometer module.
can4vscp_humidity – humidity and temperature module.can4vscp_beijing – I/O- module.
can4vscp_vilnius – AD- module.
can4vscp_odessa – Expansion module.
can4vscp_kelvin_1wire – 1-wire temperature module.
can4vscp_frankfurt_rs232 – RS-232 interface to CAN4VSCP
can4vscp_frankfurt_gprs – GPRS interface to CAN4VSCÅ
can4vscp_newyork_lcd_display_controller – Display controller
can4vscp_frankfurt_eth – Ethernet <->CAN4VSCP
zeus_expansion_roma – Ethernet controller.
zeus_expansion_moscow
can4vscp_power_injector
zeus
zeus_expansion_napoli
zeus_expansion_leicester
zeus_expansion_kiev
zeus_expansion_budapest
zeus_expansion_brussels
blitz
thunder
can4vscp_frankfurt_ble
can4vscp_frankfurt_bridge CAN4VSCP <-> CAN4VSCP
can4vscp_frankfurt_rs485 RS485 <-> CAN4VSCP
can4vscp_frankfurt_usb  USB <-> CAN4VSCP
can4vscp_kelvin_9700
can4vscp_kelvin_multisensor
can4vscp_kelvin_smt160
can4vscp_kelvin_thermocouple
can4vscp_Loos_temp_control

Paris and Kelvin NTC will be open and free as before for reference as will one or two RF modules resp.Ethernet modules.

Someone may ask why? The answer is that open source has not been working for these modules and I just don’t want to throw away the 1000h of work I put into them.