Signal K, a true game changer?
I see a lot of marine electronics and I’m hard to impress. There’s a lot of the new, faster, bigger and brighter appearing every year. However I get very enthused when I see real innovation appear. CHIRP sounder technology and WiFi enabled chart plotters are a couple of excellent examples. But now there is Signal K, and I’m really excited. This is potentially a game changer on a grand scale, and I’ll attempt to explain what’s happening and its long term implications for your boat.
Signal K is a universal data model that allows data and information about your boat, and other boats both locally and across the globe, to be easily shared. It speaks the language smart phones, tablets and the wireless Internet understand in our now much more connected world. This capability has not existed in any meaningful way before, and now it’s quicky becoming a reality.
The possibilities are nearly endless. Since Signal K can make all your boat’s data available in a consistent Internet-friendly open source format, developers worldwide would be free to create applications for your boat. This is because Signal K’s communication format makes it much simpler to access and use your boat’s information, especially for mobile devices and cloud based servers.
There are security levels to control this access, but if you allowed it, a marina could sense your boat and automatically send you docking information. Crowd sourcing of data such as water depths can be automatic. You could see not only your depth, but the depth of the boat in front of you. Sharing of your waypoints, POI’s, and even the yacht club trip route becomes simple. Check in on your boat from anywhere in the world or be notified by email if problems arise. It may seem incredible, but this is part of Signal K’s overall vision and it’s much closer to reality than you might believe. Your boat’s WiFi antenna will become in many ways as important to you as its VHF antenna.
In order to understand what Signal K is about, we have to take a look at what capabilities our boats currently have for dealing with information. Instruments and sensors gather data and typically send it to other devices using the NMEA 0183 or newer NMEA 2000 formats. NMEA data is collected, packaged and transmitted in related groups. For example the NMEA 2000 engine parameters PGN 127489 contains things like oil pressure, voltage, and also has a large number of status indicators such as “water in fuel” alarm status and others.
Your chartplotter’s software takes this data, sorts it out and displays the information on its dedicated data pages for you. It works well and nobody argues with this….. unless you want to use a mobile device and/or the Internet to easily see your boat’s data. Wouldn’t you like the limited number of wireless marine apps that are now available to do much more?
The crux of the issue is that NMEA’s communication formats understandably did not foresee the exponential growth of both the Internet and wireless communications. NMEA 2000 made its first appearance in 2001. There was virtually no wireless Internet access. Tablets and smart phones didn’t exist either. In 2014 alone over $200 billion of these smart computing devices will end up in consumer hands.
In a world where there should be hundreds of mobile apps that use your boat’s data, there are only a few. You can buy black boxes that will transmit NMEA 0183 and 2000 data wirelessly, but the manufacturers tend to use different formats and offer different subsets of the data available. This makes it difficult for app developers. They can’t just write one app and have it apply to all boats, but instead have to add individual support for different module types. Fiscal incentives for both the app developers and the gateway developers is quashed by the inherently small and fractured market. These problems can all be solved by Signal K in a very clever and unique way.
Signal K by itself isn’t designed to replace your existing navigation system. Instead it augments your boat’s potential and provides the tools developers need to create meaningful apps for your vessel with secure connections to the outside world.
Signal K is a system that takes NMEA and many other types of data and places it into a consistent and predictable data model (a schema). Think of the Web — every page has a different and unique URL. The same URL always takes you to the same page, although the content may differ second by second. Some are available permanently, some are not. Some are restricted, some are public. And most importantly you can access them from any device. Signal K applies the same concepts to your boat’s data. It defines a consistent and predictable “URL” for each individual data item.
The typical data elements shown in the schema sample all have a similar format. The information in the schema is human readable, making is easier for app programmers to access it. In the example shown it’s “headingMagnetic.” Value means this is a number and it’s currently 43 degrees. The source of the the data in this context is “self” meaning it’s comes from your boat, and “time stamp” is when the data was received by the schema so we know how fresh, or old, the data is.
The schema also stores other things important to you. Information about AIS targets, charts, routes, tides, waypoints, VHF/HF call signs, MMSI numbers, alarm set-points, cruising notes, names of crew members and much more. As the schema continues to grow, existing data types keep their original URLs, and new information types such as your sailboat’s polar data get new and unique URLs. This allows Signal K applications to keep working despite the inevitable new additions to the schema as it grows.
Arduinos can interface too
Open source hardware such as Arduinos create the opportunity to capture data and/or to manage NMEA as never anticipated. For example Signal K already has a seawater salinity data element in its schema. Arduino’s can sense this along with many other monitoring possibilities such as engine room or freezer temperatures. Lighting control with motion sensors, audio alarms like “Warning bilge high water!” and more are possible by integrating Ardunios into a Signal K based system. The Freeboard Project is completing software to allow Arduino’s to directly communicate with Signal K.
Arduinos are small, powerful and very inexpensive ($60.00) computers used to deal with the physical word. They have many inputs and outputs both digital and analog. An example of a current Arduino boating application is Jack Edward’s very successful and home-built autopilot system, which I wrote about here.
The open source Freeboard system uses Arduinos and Raspberry Pi’s to create a browser-based navigation system that uses Signal K. As part of that effort Freeboard has developed a set of specialized interface boards to aid in connecting Arduinos to your boat and Signal K. There are connections for a GPS, Peet Bros wind instrument and a water speed paddle wheel sensors along with basic autopilot functionality (a work in progress). The Freeboard Project is now rapidly approaching turnkey status. You can learn more about the Freeboard Project here.
So where is this all going? Signal K may lead toward a world where your boat’s displays are commodity items like tablets and smart phones. You select your functionality in the same way you select apps for your cellphone. Add new capabilities like crowd-sourced cruising notes. If a 15″ waterproof sunlight viewable tablet appears, just secure it to the helm, and history suggests they will appear. Waterproof cellphones (Samsung S5) are now readily available. It’s clear Signal K has game changing potential for both recreational, and commercial boating. Nothing else like it exists in a developer friendly format to accomplish this.
Navguage is an example of an open source project that has pledged support for Signal K. Navgauge takes Signal K data and uses JavaScript and SVG (Scalable Vector Graphics) to created browser-based customizable displays for PCs, tablets and phones. You can see a demonstration page showing this capability online here.
The popular OpenCPN navigation PC based software is also planning to add Signal K functionality into their next major software revision and many others are now watching the system’s progress closely.
The project has come a long way in a very short period of time. Currently a basic reference server is operational. This is a type of software prototype design, to test against and ease continuing development. The basic hardware configuration on a boat will consist of a Signal K server running on a relatively inexpensive single board processor like a Raspberry Pi, a USB hub, and a WiFi router. The NMEA data inputs can come from NMEA 2000/0183 to USB gateways, and Ardunios if desired.
Signal K implementations like Navgauge, OpenCPN, the Freeboard Project, and others are all “Open Source” projects. Like other open source platforms, anyone who can help, including commercial developers, are welcomed and encouraged to participate. You can find information about how to do this at the Signal K website.
Signal K’s goal is to help your boat become more like an iPad. One reason the iPad became such a success was the way it provided a common platform that app developers worldwide could use. It’s the abundance of good apps that made the iPad such a runaway success, not just its hardware.
In Signal K’s world apps use all of your boat’s information, not just data from sensors. Social sharing and crowd sourcing becomes possible. Also viable would be apps that do long-term data monitoring and analysis or let you use your boat’s polar data in a race. Have you selected an avatar for your boat yet?
Bill Bishop is a professional marine electronics installer as well as creator of The Marine Installers Rant
I know Bill lives in south Florida and now I wonder if he got bitten by a weird spider!
Seriously, I’m skeptical about open source projects in the tiny but complex and fragmented world of marine electronics. It’s been talked about for a long time but I’ve seen no viable results. On the other hand, the situation today is ripe, the Signal K core team looks strong, and I understand that there are some very interesting possible partners in the wings. I hope it goes well and look forward to Bill’s next report.
Ben, The spiders in Florida are big enough to eat birds and I’m still wondering where the cat went. It’s a nice warm place where everything is trying to kill you. (Thanks RHYS)
I forgot to mention that Signal K is usable commercially and only requires attribution to do so. This is a huge plus in its favor.
Ben, I only hope I live long enough! Seriously it would be a boon to marinas that have a large mooring field like Wayfarer in Camden ME to be able to send a mooring position to a boat entering the harbor. This is something I have been trying to figure out for years!
Coincidentally I’ve just been testing products from Chetco, Alba, and Maretron that all offer some of the connectedness that Bill writes about regarding Signal K. None are without issues — Maretron’s the least — but progress is being made. Check out my new BoatLogger page with boat data automatically supplied from Chetco’s HelmSmart cloud where it’s automatically collected thanks to a SeaSmart NMEA2000 to Ethernet bridge and marina WiFi. I’ll have to shut it down later today — finally going home! — but it’s live now:
http://www.boatlogger.com/gizmo?mode=Public#seasmart
Bill, Does this mean I can pull out most of the NMEA 2000 cabling once Signal K is part of all the devices onboard?
Also, I have one other concern with “wireless” and that is RFI from strong transmitters like SSB or outside signals. I’m sure developers have thought of this, but it would not be good if Signal K dropped out because of some signal interference just as I was threading through a dicey shoal. I’m looking forward to the wireless days on the horizon.
Richard, the N2K cabling still needs to stay in place. This is the data source from the sensors. The gateway/server is just another device hanging on the network. Signal K is designed to take the data and provide easy access for mobile devices, connection to other routers (land or other boat based) and the internet. The thing that makes Signal K’s design unique is that NMEA data, although very important, is just a subset of the totality of the schema’s data. The schema is designed to hold other important information about your boat such as other non-NMEA derived sensor data, cruising notes charts, photos, MMSI directories, physical vessel info (draft, length, position of the GPS on the vessel) and more.
WiFi, Bluetooth, ANT and other similar wireless technologies can all be affected by interference depending on the electronics onboard any given boat. In my personal experience I don’t encounter it very often. As an old, meaning not bold navigator I would always have a back up navigation system on board.
The Signal K team seems to have a few nice things already under their belt as evidenced by the team “bio on their wiki page. Nonetheless, they are quite geeky in the sense that what they have produced is by and large for other geeky types to admire….as evidenced by multi-step processes using make files to build and install things. The average person is lucky to spell “make” much less know how to use make. Even windows programmers have gotten used to having the build process buried by Visual C++ etc.
Presumably someone with compassion for the rest of us will “build” the product.
Cheers
Don
Panbo was not the only website down this morning:
http://www.cnbc.com/id/102177717#.
Which wouldn’t be relevant except that it’s a timely reminder about the Internet’s fragility.
I spent maybe 15 minutes looking at the protocol. General thoughts (that I’m posting here because a way to post it on their site doesn’t exist!):
1) It’s the right overall approach. JSON and HTTP is the right architecture.
2) They need to develop/standardized a very simple listen/subscribe protocol on top of this. So a client can tell a source it wants to subscribe to a certain type of update. The available things to subscribe to should be done through introspection, and supporting it should be optional. Example: client asks source “can i subscribe to you” if no reply then nope, if reply tell me what types of things i can subscribe to (e.g. wind speed updates). You don’t want to have to poll. Maybe this is already in Signal K, but i didn’t find it in 15 minutes.
3) binary data – easy to add, but you want to be able to get it – either as encoded text string or a post message.
Just my 2 cents.
Rob, if you go to the Signal K Website (link is above in the story) you will find a link to their Signal K Google groups forum. Join the group, introduce yourself, post any questions and the team would be pleased to chat with you and answer any questions you might have. There is also a lot of info in the existing topics.
Don, good comment, and the group is keenly aware that in order to be successful and effective Signal K must be accessible in both use, and installation to the average boater. This subject has been discussed in great detail. They do have compassion for the rest of us.
Maybe this is a dumb question but it’s not clear to me where Signal K resides and how to connect to it from a vessel data point of view on the hardware side and a human point of view on the other.
It would be good to see a complete end to end implementation example. The block diagram at the top doesn’t make sense at the moment.
Don,
Yes we are a bit geeky, but we do some serious sailing too, so we can speak normal english too 🙂
The code on github is definitely for other geeks. Its both working examples and basic building blocks to get developers going with their own apps.
All this gets bundled into products or software app which a user will load onto their device.
From a user point of view SignalK is a language that devices and software use to speak to each other. The difference is that its designed around the needs of the 21st century and the connected world.
outbackgary,
Signalk is a protocol for devices and software to communicate. Like a common langauge between people.
It will be built into software or devices in the same way as NMEA is.
So you will load an app onto your tablet/phone/pc and it will happily talk to the NMEA devices via a Signal K gateway (a hardware device) and to anything else that speaks signalk.
That could include your mates boat, marina, service agent, social sites, coast guard, race committee, or what have you. All with your permission of course.
Thanks for the response rob42. I have no doubt you speak and write English based on the SignalK info I’ve read. Perhaps “geeky” too poorly defined a term to be useful, and I apologize.
What I look for now is whether your abilities to conceptualize, abstract, and produce can be organized and managed and maintained to realize the promise of the SignalK proposition as a “standard” protocol and language. To me, this is when you start talking to the rest of us. Only then would I feel that there is compassion for the rest of us.
I certainly wish you the best
Don
The conundrum is that if the Signal K developers asked people like Bill not to be enthousiastic about what are doing and we refused publicity until we were all done then we will never get other developers on board. And we need those to help out both with the specification and in writing code. Maybe even a simple first end-user product?
So for non-developers, read Bill’s article as “Pie in the Sky” that shows you what could happen in the near future. Sort of like how you read about Google Glass. Ever seen one, let alone wearing one yet?
For developers and manufacturers, please come round our site, check out our preliminary specification and help us get this off the ground by joining our Google Group.
Kees,
Can I assume you are aware of the numerous open source inter-device communications protocols already in draft or better — actually implemented — that anticipate a superset of Signal K ambitions? Quite a few never took off simply because there were lots of “indians”, but no “chiefs.” These open-source efforts are a good place to recruit volunteers for Signal K. Also a good place to identify a “volunteer” project manager to capture the full and rigorous requirements set to task the other volunteers.
Cheers
Don
I find the ability to receive NMEA data through IP protocol very interesting especially if we can extend the data from the WLAN on the boat to the Internet in a simple maner.
What I haven’t understood just yet is how this differ from eg. Maretron solutions?
Or is the news here simply that signal K is an open protocol that other developers can design applications/appliance boxes for ?
Thanks
Mikkel
Don,
No need to apologise, no offence was taken. I’m quite comfortable being geeky 🙂
And you are quite right, its a long road, with many broken dreams along the way. But some succeed, and change the world.
Like sailing, you start by pulling anchor and setting course. We believe we have a sound vessel, with a solid crew, a good chart, and a star to follow. Who knows where that leads?
Rob
Mikkel,
There are a number of alternatives, like Maretron, Simrad GoFree etc. But they are owned by those companies, so Simrad is not going to commit to Maretron, and vice versa.
So the app developer or service provider is always left implementing multiple protocols. Plus you have little influence to get your own extensions.
NMEA is successful because its common to all, and not owned by any. But its still aclosed shop for the rst of us, and its boat-only data.
So Signal K is a community owned (open source) solution, where any-one can participate, and anyone can use without licence/cost etc
So a gateway to NMEA already exists, gateways to Simrad and Maretron will appear no doubt, leaving the app or service provider with jusst one simple protocol to deal with.
Rob
I’d recommend the signal K folks try to connect with Navico. Of the big 4, they seem the most forward looking re: opening up external APIs for app development (via their GoFree program).
I am aware of a multitude of different os software projects in this area and in fact the whole Signal K started on the open source nmea thread here: https://panbo.com/forum/2010/09/nmea-open-source-project.html
Don, is there something specific in your mind that you think we should consider or embrace? Please, spread the word or tell us who we should talk to.
Then again this is in no way a “must win support” case for me. I already had a working setup on my boat, but sharing the fruits of my labor (and the pain I felt!) seemed like the right thing to do.
Teppo
I think the most interesting part of this is the ability for anyone to extend the information that is passed with Signal K. NMEA is great for passing boat-specific data across a local boat network. But defining what information is included is closed and is governed by the NMEA. SignalK pushes the intelligence to the endpoints — sensors and consumers.
An earlier poster hinted at it, but one thing useful to many would be for marinas to support buoy assignment through signalK. Busy harbours with lots of marinas would make it even more interesting. A navigator could solicit the marinas in an area for mooring information and have available spots appear on their PC. For this to work would require software development for the boat side and the marina side, but all the data elements would be shared using SignalK. It’s like MarineTraffic for buoys, where and AIS message consolidation is replaced by SignalK message consolidation.
Another creative use:
Fume/vapor detectors that stream volatile vapor data over SignalK. So instead of a simple on/off alarm sounding, you get a continuous data stream.
An upward trend in concentration of explosive fumes could lead to the boat closing all fuel/gas/propane solenoids, tripping the main panel breaker, and notifying a predefined list of contacts by email/twitter/carrier-pigeon. Perhaps even broadcasting it so that nearby boats could pick up on the alarm and react. Would you like to know if your neighbor’s bilge is full of propane gas?
The exact same process could be used for monitoring stray currents in marina waters.
The central idea being that if the sensors relatively cheap and readily available, the data they produce should be even more readily available. Then you see what the market comes up with. This requires our mindset to shift from thinking of instruments and alarms to environmental sensors and data streams.
A very important step in this will be managing the data streams. How much of what information do you share outside of your boat? Who has requested what kind of information from your boat? I see the boat-router/access-point being ideally suited to this role, so more boats will install a wifi access point and router. I could see implementing a SignalK message gateway in OpenWRT and providing a nice GUI / control panel to manage the information flows. This is like the early days of the web — nobody really knew what to do with it yet, but it was the users that were inventing the applications, not the vendors. Back in 1994, people laughed at the uselessness of the idea of an internet connected vending machine that could tell you how much of what soft drinks were available, and what temperature they were at. Today, we think of this as an important tool in a proper supply chain management process.
Cheers,
/Jason
I’m glad you asked and I will be as brief as possible and then will bow out since further discussion would mean I would be obligated to start working on Signal K instead of sailing for the winter.
First, I assume something being published in Panbo is a big deal. Something “designed around the needs of the 21st century and the connected world,” to quote Rob42, is to me an extra exciting big deal. The box diagram looked like a lot of others I’ve seen, but having the all-time cynic- Bill Bishop — jumping up and down made me take notice. Mention of specifications, protocols, open source licensing, etc. sound like real substance. So…..one asks, why open source? I like open source and yet would never personally commit myself to open source if I felt compelled to change the world on this.
Second, at least to me, mention of “the 21st century and the connected world” in the context of marine electronics and software together with HTTP/JSON means what is commonly called the “internet of devices” or similar which people have based their careers on developing for both the open source community and private/commercial use. That’s why I asked how aware the Signal K community is of these efforts. Mention of the Panbo reference is insufficient. It would seem that a startup group of volunteers would seek a stable abstraction of the Signal K effort to provide a skeleton framework intended to support “the 21st century and the connected world that others have already produced and are actively maintaining and improving as a foundation for Signal K. Why reinvent the wheel. Think about how would one go about this if one were the NMEA team looking to the future?
Third, “the 21st century and the connected world” sound like a “deliverance from evil.” While this might prove to be true and I find myself praying on its behalf, I also find myself shuddering thinking about security. Admittedly, I’ve been a security “freak” since I first hacked (for demonstration purposes only) a dual port printer shared by an ultra-high security facility (yes I would kill you) and a public facility in the 70’s (fast printers were not cheap). How is one going to address this? Hacking is especially easy with HTTP/JSON. Defending against it is 24/7. One should address this before writing any code for the connected world. How is one going to ensure that the code one distributes via open source is clean? How will one become the de-facto standard for marine devices without demonstrating one has crossed all the security T’s and dotted all the I’s? How do successful open source communities such as Mozilla go about this?
Fourth, how is one going to organize such that the open source product survives the interest of the initial developers? What environment should one have that allows new developers to join and contribute, while maintaining code security, scope and schedule?
Fifth, how should one validate code? Should there be test protocols before code is written? How else can one intercept unintended results? I ask anticipating that my life could depend on it. Is the current collaboration platform rigorous? Does one need to check code out and in? Are test protocols valid?
While I anticipate the above items are thought provoking, and I could go on and on and on…., I’m not being fair since I have nothing to lose regarding Signal K. so, ultimately, the only fair question which I would ask is how much skin does the team have in the game? Is one a mutual admiration society or is one truly viewing this as a “do or die”. In the former case, I apologize for raising the issue. In the latter case, keep us posted.
You are all wonderful individuals and you have to decide.
Spot on Jason!
Another one that interests me is harbours with bar entrances. If each vessel pushes its type, draft and track with depths to a local server using signal k, then when you approach an unfamiliar bar you can grab an overlay of recent tracks and display that on your chartplotter.
Thats a lot more useful than a chart or cruising guide as they date so fast because the sand on the bar shifts with every storm.
There are a number of interesting anchorages here that are little used because no-one is sure about the entrance.
Rob
My day job is international standardization of database query languages and a whole bunch of stuff in that arena. I don’t disagree wit the statement that JSON and HTTP are the right tools (although my personal preference would have been XML over JSON simply because the toolset for working with XML is much larger and more mature).
The principal concern I have with JSON is the belief by so many of its fans that “we don’t need no stinkin’ schemas”, which fans of so many other data representation protocols have learned is a path to chaos and failure.
In particular, looking at the “Schema Data Format Example”, it’s obvious that we’re looking at a portion of a JSON object. What is less obvious to many people is that the name of that object “headingMagnetic” MUST BE NORMATIVE in order for anybody else to know how to find and use the object. The same is true of the numbers, names, and data types of the fields in that object. If other projects using Signal K just make up names and structures “on the fly”, then others will not be able to use the information.
I’m sure that the Signal K people understand this, but they must also make sure that their potential audience understand it as well.
But we do need a fabulous schema and indeed there is one, like Bill’s article already mentions.
For humans:
http://signalk.org/specification/index.html#schemas/signalk.json
For machines:
https://github.com/SignalK/specification/tree/master/schemas
Validation tools:
https://github.com/SignalK/specification#validation
and for example the n2k-signalk converter tests assert that the output must be valid SignalK according to the schema.
Interesting.
Is it two directional, can you write information back also, for example configure your boat sensors?
If a component manufactured allow it, does the specificatoin provide for updating the firmware in sensor, displays, and other devices ?
Dan,
Definitely 2 way. Its designed to be fully peer-to-peer, although in practice a lot of sensors will be tied to a small shared server, as many sensors wont have the ‘brains’ to be a full peer.
Updates to firmware or software in general are technically feasible but we have not looked into the matter yet. Since every device is likely to have a different process to upgrade it would likely be some sort of command to upgrade, with custom info on how. There are a number of precedents in the software world (eg android apps).
We would also need to consider the security – I dont want my devices upgraded except by explicit permission, and from an authentic source. But again thats been done in a number of environments already, so its certainly feasible.
BTW I’m adding the idea as a ‘user requirement’ in the forum.
Rob
Nice! NMEA’s relatively new President & Executive Director Mark Reedenauer just posted “a friendly hello from NMEA” on the Signal K forum along with the offer of “technical resources if NMEA 0183 or NMEA 2000 questions arise during the development of Signal K.”
https://groups.google.com/forum/#!forum/signalk
A positive step. Next questions from Signal K likely to include … can we have access to the NMEA 2K specs.
Sparky, I guess we’re pretty predictable.
We also have an interesting HW partner that’s hopefully going public soon…