Shouldn’t our community sourced marine data be open to all developers?
I’ve been cogitating a lot about crowdsourced depth data lately, including the realization that “community sourced” is a better term. Whatever it’s called, Navionics in particular has made it wonderfully easy to collect and share sonar files and especially wow with the Vexilar integration. But the business stakes are high and thus we have the frustration of Navionics and Garmin butting heads. Upon further contemplation, a wistful thought from that last entry — “Wouldn’t it be great if we could upload our data to some service that would make it available to any chart developer?” — seems not only important to avoiding further messes and helping this technology proliferate, but also quite possible…
The “ownership” of community sourced data is an issue. Sure, we physically possess the sonar files we make ourselves, but very few of us will go to the trouble to send those files to every chart maker that could use them. In reality, the data will amass in the servers of whichever company makes it easiest to use and to submit, and that company may eventually “own” the category. It’s well known how this “the rich get richer” cycle can work with crowdsourced data — think about how our product reviews helped with Amazon domination and how our search data benefited Google.
So one reason Garmin management may be reluctant to work with Navionics SonarCharts, besides the integrity of their own tight chart/MFD integration, is that they don’t want to help Navionics acquire ownership of community depth sourcing. Similarly, while Navico does support SonarChart display (though only as fishing maps, I think), they do not make it easy to collect soundings for Navionics with Lowrance, Simrad, and B&G gear. There’s a competitive tension underlying data collection that works against the overall community. It’s fine with me if lots of chart makers compete over how to distribute and display community sourced data, but I want to share mine with all boaters, not just users of one type.
Therefore I think we need some organization willing to be a neutral Switzerland of community sonar logs, able to accept depth data in many forms via many routes, store it securely, and make these raw files easily available to any commercial or noncommercial chart developer who wants them. OpenSeaMap seems like a perfect candidate, TeamSurv also seems like a possibility and there are, no doubt, others.
As a subproject of OpenStreetMap, OpenSeaMap looks like an ambitious open source effort to chart the world using community sourced data. The OSM volunteers have already created a good sonar file collection infrastructure — including the inexpensive hardware logger seen above — and they collect data under an ODC Public Domain Dedication and License (PDDL) that seems right for maximum sharing.
I want to be perfectly clear though; I am not suggesting that Navionics and other commercial sonar charting companies simply use community sourced depth data processed by OpenSeaMap or any other third party. The chart developers should be able to process our files any way they like. But I am suggesting that OSM or a similar open source organization modify its model to give all chart developers easy access to our raw data files. The result could be a lot more data for their own project, and doesn’t the whole concept fit with the open source spirit?
Last weekend I registered as an OpenSeaMap depth data contributor and was impressed that I could set up different (metadata) profiles to describe different boats or equipment setups used to collect the data. OSM even makes it easy to describe the relative locations of a depth transducer and an associated GPS, so that the usually small location/depth discrepancies can be removed in processing.
I uploaded several of the file formats that OpenSeaMap already accepts, and it was neat to see them get processed and viewable within an hour or two. There seem to be many depth file formats in use, many of them proprietary, but I’m hopeful that the open nature of the proposed raw data pool will encourage developers to divulge their specs (or other developers to reverse engineer them ;-). One issue might be full sonar files like the one above called “CAMDEN.sl2” that I recorded with a Simrad system. The 58 megabyte file covers less than a mile of bottom track, though Insight Genesis can use the extra information to determine bottom hardness and vegetation types. I suspect that bulky file size may be an issue even with Navico’s slick new GoFree MFD data uploading, but for the most part depth files are efficient and the components are so generic that they can be generated in all sorts of ways.
The Garmin HomePort screen above shows a GPX track file I created on a Garmin MFD, but it could have come from Navico, Raymarine, or many other sources. The 1.2 megabyte file covers 63 miles of Delaware Bay and vicinity at fairly high resolution — Navionics recommends a 1 second time period and collecting speeds under 15 knots — and I think it contains all the data fields needed for proper sonar charting (depth and position are obvious, and time is needed to correct for tide levels).
Here’s what that GPX file looks like in OpenSeaMap’s track viewer and also when overlaid on actual OpenSeaMaps. The presentation needs lots of work! It should at least have adjustable color coding, so that anyone shortcutting across that particularly tricky section can see green where it’s safe for their boat and red where it’s not. But this whole open raw data pool idea is agnostic about presentation. I’d be happy to see OSM make better use of my depth files, but it doesn’t really matter as long as every other chart developer can also use it.
In fact, OpenSeaMap needs an enormous amount of depth data to accomplish its goals. I significantly expanded its North American database with just a few files (open the map and check View/Water Depth beta to see what I mean). But I’m hoping that’s motivation for OSM to adopt the raw data pool idea, and that readers will add further encouragement.
Not that OSM, or TeamSurv, or anyone else could make this idea work all by themselves. While OSM has already built good tools for collecting community depth data, Navionics in particular has taught us all that making it really easy to contribute data is highly important. So, it’s encouraging that founder Giuseppe Carnevali seemed encouraging about this raw data pool idea, and I’d certainly appreciate the Navionics SonarCharts even more the company works with an open pool and especially if it makes it easy to contribute data to such a pool. And ditto for Insight Genesis. Plus it seems reasonable to think that developers who aren’t yet involved with community sourced depth data might help collect it to a common pool. There are all sorts of charting and other apps that can already take in depth, GPS, and time data from my boat systems via WiFi, and wouldn’t it be nice if they could automate depth file uploads?
But maybe I’m having a fever dream here? I usually write about technology that already exists, not wild ideas about what could exist. Please deliver your feedback below.
I’ve heard boaters say that they wouldn’t contribute to Navionics SonarCharts because Navionics charges to view the processed data. Isn’t that another reason to open source the data contribution side of things?
Incidentally, readers, I wasn’t completely joking about a fever dream. I’ve been house bound for a solid week while recovering from the replacement of my left hip joint with some nice titanium, stainless and porcelain hardware. Everything is going very, very well but there are quite a few drugs involved 😉
You hit it right on Ben. I have a problem supplying without compensation, crowd sourced data of any kind to a company that sells my data. Either compensate the originators (perhaps by supplying free equipment as does ARGUS), or make the data freely available to everyone (as does ActiveCaptain). Failing either of these, I will keep my data to myself.
The whole crowd sourced chart data thing as it is evolving is fraught with problems. You can’t have crowd sourced data without a crowd, and neither Navionics not Garmin are saying how many boats are supplying data, or where they are. Therefore, I have to question if the charts sold by these companies (and some others) are not just government charts modified to look a bit different, or a figment of someone’s imagination.
I believe that ultimately Crowd Source will be a huge improvement over relying on sparse government surveys. I like the concept of some sort of central depository for data. Hopefully someday everyone will get their act together, and improve their credibility.
Smells good Ben. I’ll definitely take a look at OpenSeaMap. Only concern is that the imported data be anonymized.
Let me take this opportunity to wish you a speedy recovery 😉
The best data will be from boats with quality, well installed systems from the major manufactures. Voices like yours that make the case for the value of community data being for the community will be one of the strongest influences on the manufactures. Your point that it needs to be easy and automatic is essential and long term data from a fishing boat with a shallow keel and good local knowledge will identify a shoaling pattern early in the season or even the same day that provides a warning for sailboats later in the day. There are certainly huge issues in keeping the quality of the data high and filtering out or flagging questionable soundings (can a boats GPS signal be degraded or a clock off?) But as us internet denizens like to say, data wants to be free, and that will be good for everyone.
I hope you continue to consider and write about these issues. Good luck with your new hip, and keep up the PT. 🙂
I do wish the black box data logger was a direct NMEA 2000 plug-in. This way it can collect whatever it needs off the N2K network to make the most out of my collected soundings – so much easier then dealing with 0183.
Also, how do we know if each of the community contributors has a calibrated depth sounder, (programed the correct offset)? I’m sure this has been considered already.
I’m fine with contributing my soundings to the community without compensation. We all have to pay for commercially made charts. I would rather know the soundings are accurate – it’s a safety issue not really a money issue.
Terrific idea, as long as it’s not free.
At some point along the chain, several parties need to get a return on investment. The consumers need to get a return on investment, the data collectors, and the fine minds that crunch the data. Return on investment can be money and/or ego snacks.
Consumers: Getting consumers to pay something for the use of the information, especially consumers who don’t contribute data, would help the rest of the supply chain.
Data Sources: The investment (boat upgrades, labor to get files off boat) to give my data away is important, not for myself, but that its a business model that gets many quality contributors. Maybe some contributors should receive money, e.g. fishing boats that go out every day perhaps making a profit at giving their data, and for volunteers they get free public recognition for their contribution as an incentive / ego snack for their work (like ActiveCaptain).
Data Crunching: There is undoubtedly a bunch of effort that goes into this. For example, creating all the algorithms and number crunching to figure out that my boat’s offset is X inches, and that it changed mid-season by comparing my boat with others over the same point of land at different points in the tide. Perhaps data collected from my boat isn’t even usable until I have been out enough to cross over the positions of three other boats? Is it reasonable to get out shoaling data same day, when the boat that is the source of that data has not enough data points to compare with other boats?
Does someone have a dream to make this all happen?
… or can we keep Ben on drugs a bit longer ?
There should be a heatmap kind of layer that shows data quality or confidence. A mix of number of soundings in a given radius, min/max/mean values. Also, data sources need to be given a reputation score according to the reliability of their data. This could be a dynamic value where the data provided by a boat is compared to the existing dataset.
Data consumers can then extract only the data that meets their preset criteria, such as number of soundings available near a given location, age of soundings, reputation of anonymized data source. If there are only two sets of data for a given pass, I may be much more careful in trusting that data, however, if there are data points from hundreds of boats over the last few months that are all within a small margin of error, then I can be fairly confident that the mean value is representative of reality. The user should be able to define various thresholds for using values, like not showing values for data without x number of soundings, or that vary by more than x feet. Basically if the data don’t converge to a sane value, don’t show it.
But this is the great thing about crowd-sourced data. You can either consume a processed stream or use the raw data to do your own processing, such as normalizing the data to account for tides, waves and surge.
I fully agree that ease of contributing data is key to any project’s success. Make it easy to feed your data and make it free to access the raw, anonymized data. Processed streams can be provided as a value-added service, and real market capitalism will drive prices to competitive levels. Manufacturers must be convinced to open up their devices to these open data sources. Where the product differentiation comes in is in the processing of the raw data. I have no problem with paying a small amount for data curating and processing. Different vendors will massage the data with different algorithms to arrive at ideally the same result for the same timeframe, since the ground doesn’t move… much.
An interesting feature could be identifying on the chart where community data diverges from official chart data and local sensor data. This could help keep skippers alert to potential chart errors.
Dan,
I think once you make this into a business model and pay people, fishing boats, for their depth data you will double the price for electronic chart regions. The price at the end of the food chain has to go up. For cheap sailors like me, as much as I like technology, I’ll go back to paper should that happen. I did fine with paper charts for over two decades, so no problem. If ActiveCaptain paid people for the information they posted I don’t think you would find AC on many navigation programs – it would be too expensive to sell the service. Getting mariners to contribute freely is a little like public service, it helps everyone navigate safely – it’s for the greater good.
@Richard C.
I had the same reaction to the lack of direct N2K support for the black box logger. Probably due to the NMEA Licensing.
All the more reason to push the manufactures for complete support for open data formats like GPX (w/ standardized depth extensions). Yet another reason why something like SignalK is a benefit to the end users.
Geez, Dan, you’ve made this idea much more complicated than it is. It’s only about gathering and pooling the raw depth data files, not the whole chain of events. Crunching the data, charging for it, etc. remain just as they are today, which is completely up to the developer.
If OSM wants to give away maps with my data in them, fine. If Navionics wants to mix my data with all sorts of other sonar sources and charge for the result, fine. Etc. etc. The point is I want to submit one data file to one repository that every developer can access. Simple as that.
Well, there is a bit more to this project than just holding the data files. The raw files I uploaded to OSM were already anonymous, but any developer using those files needs at least a bit of the metadata I supplied via OSM registration and forms. The main thing is transducer depth, but some of the other stuff like transducer model and GPS offset might be useful to whoever does the data crunching. Obviously OSM should not also pass along our names but I think the filtered metadata part is pretty trivial.
Jason, I thoroughly agree that community sourced soundings and mixes like SonarCharts could really benefit from a “confidence” layer, and I gathered in Miami that Navionics has heard that request from numerous users. But again that’s a data presentation issue that’s completely up to the developers and not part of this raw data pool idea.
I think whatever database keeps a central copy of the sounding data should require people to create a user account so we can keep track of what sounding data comes from which users. That way, after the fact, the quality of individual sounding data accuracy could be voted up or down (3 stars for accuracy or 5 stars?). Was the dept offset correct? etc. could be determined after the fact by a simple rating system. This rating system could/should also be public.
data tracked could simply be:
lat, lon, depth, timestamp, userid
Then the userid and timestamp could be used for all sorts of corrections after the fact.
(I’m in the database business, hence all the details here.)
I just don’t know how all these layers would mix in a live-boat situation. How do you display on a
But, Robert, in this proposal there is no central database of depth data. It’s just a repository of the raw data files sent in. In the comments to the previous entry Giuseppe Carnevali said that this is what would work for Navionics and I get that. They want the exact same user files they’re already used to crunching with no processing by a middle man. All the repository has to add is access to the metadata with a user id (as you suggest) instead of a name. The ID could be in the file name and the anomyized metadata database open to developers.
(Incidentally, Navionics already gets a user ID and some metadata because you can’t upload files without signing in and answering a few questions like transducer depth. I dare say all the companies that accept depth files work the same way. So to put this repository idea a different way, it would look to the chart developers almost like what they’re already getting from data collectors.)
I have not yet heard back from OpenSeaMap (and it may have been rude and/or impetuous of me to post all this without getting in touch first) but if they do this like idea, I’m pretty sure that they would be parsing every file format they could figure out into their own depth database. And maybe they’d be willing to give that away separate from their whole map packages, but that’s completely up to them.
I’m not sure your idea of voting on individual data sets is workable but, again, that’s on the data presentation end of the chain. The idea of the pool is give all developers as much raw data as possible but no encumbrances on what they do with it.
Careful, Jason, you may be getting my fever !-)
For instance, the Chetco SeaSmart black box I started testing last fall already collects all sorts of N2K data and pushes it up to the Internet for logging, engine analysis, etc (or saves it to an SD card until it’s online again). Setting up the system is not for the faint of heart but I bet the developer could easily add a feature whereby GPX depth files were automatically forwarded from his HelmSmart cloud to the proposed depth data repository under my account. Total automation.
And how about that black box in your pocket? I have all sorts of iOS and Android apps that can already receive my boat’s depth, GPS, other data from Navico GoFree or Vesper XB8000 AIS transponder or Coastal Explorer (via boat router) etc. It doesn’t seem that hard to add a feature that creates depth files in the background and posts them to repository when done.
As far as discrepancies go, that’s exactly what a good navigator is looking for!
In my case I have being using a DMK box to consolidate all data generated via SeaTalk (depth, gps, wind), and VesperMarine (AIS) and send it via Wifi to iNAVx navigation running on two iPads and PolarView navigation running on the MacBook. It breaks my heart, every time, seeing all the depth and gps data stream by with no way of capturing in either IOS or OS X.
All it would take is to capture the UDP stream, containing the data being sent by the DMK box and save it. The resulting file could then be processed by a utility to create an open standards compliant sonar log for both my use a and to the boating community at large.
In this case no black box is needed. If anyone knows of apps that can log and process the UDP stream logs, please let me know! I would be an eager beta tester!
I feel your pain, Paul, but I don’t think you should have to mess with more software. If this project really happens, I’d hope that iNAVx or PolarView, or both, will code a depth file collecting and uploading feature. After all, they too stand to gain, first by making you a happier user and second if they decide to use community sourced depth data available from a successful open source pool.
I freely share ANY boating knowledge I may have with my fellow boaters. If anyone can turn what I give freely into a profit good for them.
My view is that boaters are a family of diverse people sharing an affinity for being on the water.
If I could save one boat or person from the shoals it would make me a happy man.
Ben,
I think Dan hit the nail quite well regarding the next steps beyond the collection of data.
While I share your enthusiasm for the collection of data, I’m equally leery of using such data without the number crunching necessary to filter errors, especially in shallow water.
A large data set can be wonderful input and the OpenSeaMap has a way to consistently “calibrate” initial input. Nonetheless, I suspect Dan has barely touched on the number crunching necessary to normalize all the inputs to be at the level of a government-based e-chart.
In response to Butch, please remember that someone has to run into and hopefully over the shoals to actually measure them. Like Ben’s mapping project in New Bern, I do this from the dinghy. As far as I can tell I have one of a few dinghies in the Caribbean and Bahamas with a logging depth finder/MFD.
Cheers
Don
Hi all,
Ben’s just pointed me towards your post, and TeamSurv is ready to accept all of your data – we already have 200 million data points and a bit under 300 loggers, with the volume of data doubling every year at present.
We supply a hardware logger on free loan that takes data from NMEA0183 and Seatalk, or you can upload log files from Garmin or Navico. Many software apps also support us, see Info->Nav Software on the TeamSurv web site. We’re starting work on N2K later this year.
As well as getting data from leisure boats, we also get data from research vessels, and are working with the IHO on their plans to get commercial shipping logging data (though as with all international bodies this is inevitably a slow process).
Regarding data ownership etc, when you upload data to us you licence us to use it to produce things like depth data, but we keep all your personal data confidential so nobody can associate a track with you or your boat. We make the processed data available free of charge to all of those contributing data, but we reserve the right to charge others, especially commercial users, so we can cover our costs (we’ve had some grant funding from the European Union and European Space Agency, but this doesn’t cover everything).
Our preference is to make available the processed data, not the tracks. Processing the data to get good results is not trivial, and is beyond the capabilities of most nav software developers. For example, we correct for tides using a mix of tidal station predictions inshore, a global tide model offshore, and further corrections using actual tide gauge data; we correct for variations in speed of sound due to water temperature and salinity; for ships we correct for different load conditions and for squat. We then process all of the tracks statistically to get the depth data plus a number of data quality metrics. This is far more than OpenSeaMap or Argus or Navionics do, and it shows in the quality of our results when validated against a professional multi-beam survey of the test area.
Anyway, you’re all welcome to sign up at http://www.teamsurv.com – first register and then click on the Join In button.
Tim
Sounds good, but…. A couple things come to mind.
What could be put in place to verify the initial info of installed equipment? Im confident all panbo readers could fill out the information accurately. What about those that had the yard or a friend of a friend install the equipment a couple months or years ago? “Yeah, 3ft sounds about right and I think I added a extra foot in the setup process, just to be sure”.
Who would be responsible to verify the tide and or meteorological conditions at the time of the sounding? Was there a offshore system that pushed a bunch of water into the cove?
None of these issues are insurmountable but the result would only be as good as the input. Not having something in place to independently check the source would cripple the process from the start. Even with a process in place the amount of information required to get a clear picture will take a long time to compile, especially when you talk about out of the way, shallow, low traffic areas.
I like the idea, it would be another resource to check when entering unfamiliar waters. Without a solid foundation though I think we would wind up making a big circle. Carrying a bit more rode is the best defense against shallows but there are times one may need or want to get “just a little closer”.
Thanks, Scott and Don, I understand your concerns but again they are irrelevant to the data pool idea. All the entities already accepting sonar files have the same issues and to the best of my knowledge they all handle them the same way: they hope that the transducer depth and other metadata submitted by the contributor is accurate — since there’s no practical way to check — and figure that bad data will eventually reveal itself as more files for the same areas come in. Navionics, Navico and others may also compare user data with existing bathymetry files and make some assumptions about which is right, or do some averaging. We really don’t know how they’re processing and mixing data. But consider this:
Maybe this data pool idea actually works out and many of us contribute depth info to a pool from which almost all the chart developers draw (sorry, Tim, it has to be an unprocessed pool, at least for the big guys). Then we get to see and compare how our raw data is used by many independent developers. Isn’t that ultimately the best way for winners to emerge, instead of one or two companies dominating because they collect the most data first?
Ben, Remember that Argus Survices has been collecting this data for many years. We have been a contributor to Argus for some time. Much of the information is available through their website but not displayed in a fashion I would prefer or that most boaters would consider using. But perhaps with some encouragement, they could be convinced to jump in and revisit this discussion. Chuck Baier
Thanks Ben for this great post. I agree with the others that contributed data should be free for everyone. If that is guaranteed, I’m happy to contribute.
Also I’ve been playing with the idea of getting a dedicated data logger for a while now. The box you link to offers great value for the money, so I just ordered one. I’ll hook it up to the NMEA 0183-output of my Vesper XB8000. Lets me log my tracks in a very reliable and energy-effient way and contribute the data to OSM.
Ben, I like your idea of all the data being open, but the question then is how does the service meet it’s costs? What I can see happening is that the “independents” would log data which would be sucked up by the big guys with nothing in return, and the users of Navionics, Garmin, C-Map, … would still be paying the same for their charts. Meanwhile, the independents find that there is no revenue to keep going, and they ultimately fold.
Unless there is a clear way to resolve this, I think a model like TeamSurv’s is more sustainable, where all contributors get free access to the data, whilst we reserve the right to charge others (particularly commercial users) for the use of the data. This way, those putting something in get free data, whereas those that aren’t contributing data make a contribution to costs.
Ben,
This interesting discussion is about one basic issue: there are at least a dozen companies around the world trying to crowd-source sonarlogs to make charts from them, but in the process the sonar manufacturers are creating all possible barriers to prevent anybody else from gaining access to the data that boaters collect with their instruments, and this is detrimental to the common good.
Therefore your idea is: can we shift the competition from collecting and processing the data, to processing only, while the collection is free of barriers from the instrument manufacturers because it is done to a community-owned repository?.
We support the idea, but the practical answer seems to be “no”: it is clear from the blog that not a single hardware manufacturer is willing to give up on trying to “own” the data that are collected with the instruments that their customers bought from them, and all the organizations that I know, including TeamSurv and OpenSeaMap are just as intent as the others in competing to collect the logs without sharing them with anybody: the common goal is to output charts, not logs.
In this scenario, the best that can be done is get boaters to reclaim their right to freely share their data with however they choose, one or many, and to use all the charts of their choice. This is for the common good and safety.
I can see a value in the idea of Community Sourced data but I caution that there are still issues. I had a new sonar installed and for several months I would get regular incorrect depth readings. How do I know? I KNOW our harbor is not 900 feet deep (at most 25). I also know that 5 miles off Channel Islands Harbor it is not 5 feet deep.
While trying to figure out and fix this issue, I was recording these depths and when I would go to update my software, It would request to upload my data! I never have uploaded for this reason.
User beware still applies!
Giuseppe – why do you say that OpenSeaMap data is not available. The data shared with that project is under Open Data Commons. See this from their Wiki:
What license is OpenSeaMap under?
All the data in the OpenStreetMap database, including OpenSeaMap data, is under the ODbL. The chart tiles are under the “Creative Commons Attribution Share-Alike 2.0” license. In short, the license stipulates that any use of OSM data is permissible, even for commercial purposes, however OpenStreetMap must be attributed and the resulting data must be released under the same license.
From my read of the simplification of odbl
http://opendatacommons.org/licenses/odbl/summary/
That means
You are free:
To Share: To copy, distribute and use the database.
To Create: To produce works from the database.
To Adapt: To modify, transform and build upon the database.
As long as you:
Attribute: You must attribute any public use of the database, or works produced from the database, in the manner specified in the ODbL. For any use or redistribution of the database, or works produced from it, you must make clear to others the license of the database and keep intact any notices on the original database.
Share-Alike: If you publicly use any adapted version of this database, or works produced from an adapted database, you must also offer that adapted database under the ODbL.
Keep open: If you redistribute the database, or an adapted version of it, then you may use technological measures that restrict the work (such as DRM) as long as you also redistribute a version without such measures.
– See more at:
http://opendatacommons.org/licenses/odbl/summary/#sthash.vhAeHJ6z.dpuf
So it seems to me that the underling sonar DATA is open….so long as your derived works are OPEN too.
Thanks, Adam, but that license won’t work for Navionics or any other commercial chart maker that is trying to charge for their processing and distribution work. It doesn’t make sense either, as OSM would not be adding value to the raw data files, just holding them in a pool.
The ODC Public Domain Dedication and Licence (PDDL) under which I submitted my raw data to OSM does seem to make sense, as it doesn’t constrain the chart developers in any way, like having to redistribute their processed database for free.
But I’m afraid the whole matter is moot for now. OSM as it stands has virtually no depth data, despite building some nice collection tools. It’s hard to collect collectors!
Moreover, they seem to have no interest in my raw data pool idea. Early this week I emailed every OSM email I could find, including an apology for dropping this idea on them in such a public way, but I haven’t gotten back a peep.
I’m embarrassed but also have some other ideas about possibly interested parties. As Tim noted there is no obvious payback for providing an open raw data repository, but there is good will to be earned and I understand that even “free” sources like OSM are sometimes voluntarily subsidized by commercial users.
It’s also good news that Navionics supports the idea of competing purely on processing the data, not also amassing the most data first. Thanks, Giuseppe!
Ben – I dont know..I’m a bit more optimistic about derived works. Open Source allows you to make money w/value add tools capabilities and services. Navionics is MORE than just the charts and processing…but also the support and warranty, the packaging, the ability to get other tools like planning and interfacing to various plotters.
Otherwise, the only way it works is if there is a governmental body that is a pool of data/sonar information.
My customers are primarily commercial fishermen. They all want to do bottom mapping. Currently there are two programs that do it. Olex and Nobeltec. These options are cool, but none of my guys will ever want to give up their data for other people to see. I don’t blame them. It’ll be pretty great when one of these manufacturers come out with a plotter/sounder that will record and build updated charts without the need to share them.
Hi All,
I’d consider myself a member of the openseamap project. I mostly map seamarks, hazards to navigation, and got recently involved in the making of the Garmin maps. While not being involved with the depth data subproject directly, I’d like to clear up a couple of things.
Basically the only difference between Openseamap and all the other crowd sourced bathymetry projects is the license. Even the TeamSurv project is under a non- free license (I use the term free in the context of “free as in speech, not as in beer”).
The TeamSurv website clearly states that:
“The charts are for your personal use only. You may not distribute, reproduce or publish the charts or the data therein, whether for commercial purposes or not, without prior written permission. If you know someone who would like to use the charts, please ask them to download the data from the web site for themselves.” from: http://www.teamsurv.eu/Data/Charts/tabid/257/language/en-GB/Default.aspx
This means that by contibuting to the TeamSurv project, you basically give up your rights to your data. Nothing prevents TeamSurv from closing up the data in the future, for example by only making it available commercially (just like Garmin and all the others). Please note, that I am not implying that TeamSurv has any actual intentions of really doing so.
Only with a free and Open Source license one can be sure that the rights to the data stays with the community. Even for folks like Garmin and Navionics, although they probably do not contribute to the OpenSeaMap.
As I said earlier, I am not directly involved with the depth subproject, but my guess of why the raw depth data is not directly available probably only has to do with resources. The manpower of the OpenSeaMap project is quite limited (hint: volunteers very welcome!). For now, they really concentrate on calculating depth contours from the raw data. If you have any questions you want to ask the depth guys, I suggest you go through the OpenSeaMap forum or, even better, the mailing list.
http://forum.openseamap.org/index.php
https://lists.sourceforge.net/lists/listinfo/openseamap-develop
Christian,
I think you’ve got a bit confused. The quote you give re TeamSurv is from the chart download section, not about contributing data. We state it because first we want people to download data from ourt site rather than pass round old products, and secondly as we make quite clear we charge for commercial use to cover our costs (e.g. if Navionics wanted to use our data and include it in a commercial chart).
Regarding contributing data, the agreement is quite clear: you continue to own your data, and you licence it to us to use for creating the maps etc. Products with TeamSurv data will always be free to those actively contributing (though if they incorporate data that we have to pay for, we may charge to cover that). Personal data is kept confidential. At the moment data is free to everyone for non-commercial use, though in time we may make a small charge for digital charts (e.g. Antares charge £20/year for their detailed charts of Scottish waters).
Adam,
With OpenStreetMap, CloudMade was set up by Steve Coasdt and Nick Black, founder and long term supporter of OSM, and they failed to make it succeed – it now just does general mobile mapping tools.
With the much lower number of both users and commercial mapping organisations in the marine sector, I just fail to see how this approach could work.
The IHO (the umbrella body for national hydrographic offices) are talking about setting up a repository for crowd sourced data, but this will primarily be for ships. Also, they are working on the assumption that a number5 of organisations will volunteer to act as hubs to recruit the ships, supply data logging equipment, retrieve the tracks, subject them to quality control etc for no payment by either the IHO or the shipping companies, and it isn’t clear to me why any organisation would want to do this, especially as shipping is obviously much more commercially driven than the leisure marine sector.
Hm, I still think I got it right :-). When I said “non- free” license I meant free as in speech, not as in beer (as I clearly stated in my post).
Under what license will the whole TeamSurv database be? OdBL?, Public domain? I dind’t find any specifics on the TeamSurv site.
It might be that contributors still own their own data, but a couple of GPS tracks with depth soundings in them are next to useless. Only when a certain track density is achieved, the value of the whole dataset is much more than the individual track. This exactly why crowd sourcing is so great.
In the end only TeamSurv will be able to use the database as a whole. It is owned by TeamSurv, not by the community.
I personally have no problem with TeamSurv trying to recover their costs. Nor do I want to imply that the current TeamSurv crew has any intention of closing up the database at some point in the future, but there is no guarantee for the contributors that it won’t happen. What for example happens if TeamSurv will be bought another entity (Garmin, Navionics)? TeamSurv is not an “open source” project. The fact that the resulting data is available free of cost to the contributors doesn’t make it so. Open source is not the same as free of charge.
I might be terribly wrong about this, but then please elaborate a bit more about the type of license you guys are using for the database and how you are planing of making it available to the community.
Best Regards,
Christian
Thanks, Christian. I also heard from Markus Bärlocher — “the initiator of OpenSeaMap” — who wrote:
“Thanks for thinking about a worldwide system of free hydrographic data! 🙂
The solution of OpenSeaMap is “Water depths by crowdsourcing”.
The challenge is to build a system like Wikipedia,
in order to *make the world really better for everybody*
And to avoid that big market players abuse the idea for their own aims to become “the richest of the rich ones”.
We believe in friendship at sea and help each other 🙂
We believe in synergies and in the freedom of people.
It is good if people are self-determined and self-responsible.
We don’t like the “doner-war”, where the old shop sells the doner for €2.50 and a new one sells it for 1.50. The old one now needs to sell it for 1.50 too – and dies. And the new one now sells his doner for 4.00! 🙁
We don’t like attempts to lock out others by artificial restrictions. It is not a good idea, to need a different power supply for each device, or a proprietary connector for each interface.
We go for open protocols like NMEA-0183 or CAN etc.
We use also other well known protocols.
{your open data entry} Looks nice 🙂
Do you think, OpenSeaMap should be the “owner” of this
“Neutral raw data pool”?
How can we prevent, that commercial organisations take the data without contributing data?”
Markus also sent this article he wrote that was published in March, 2013:
http://www.hydro-international.com/issues/articles/id1444-OpenSeaMap__the_Free_Nautical_Chart.html
And this summation of the OSM depth project:
http://wiki.openstreetmap.org/wiki/File:OpenSeaMap-Flyer_Waterdepth_Skipper_en_web.pdf
I’ll add my comments soon.
Christian,
TeamSurv is not a fully open source project, and has never claimed to be.
Being fully open source is unsustainable, in my opinion. Otherwise why would Markus ask (presumably rhetorically) “How can we prevent commercial organisations taking the data without contributing it?”.
Either you put a strictly no commercial use licence on it, in which case the project will not scale beyond technogeeks who sail, or you accept the realities of the world and charge commercial users so that the project can scale and can offer a good product to all contributors of data.
Even with the much larger number of land based users, OpenStreetMap is run on a shoestring. For marine use the area to be covered is twice as large, but the population is much lower, so scrabbling after donations from users of your data who happen to have some spare cash to give you is unlikely to work.
Thanks to OpenSeaMap for getting in touch, but I have to admit disappointment. First let me address Markus’ questions:
“Do you think, OpenSeaMap should be the ‘owner’ of this ‘Neutral raw data pool’?”
I don’t think that the term really applies. I want to submit my data to a pool where it just sits untouched but available without constraint to any non-commercial or commercial developer that wants to use it. In effect, it would be held in a trusted pool but “owned” by all.
“How can we prevent, that commercial organisations take the data without contributing data?”
Sorry, I don’t understand. This idea is about getting as many boaters as possible to contribute depth data to a common pool. It has nothing to do with developers sharing data with each other, except that if it worked well they wouldn’t have much unique data to share or trade.
Markus, I mean the following as a friendly jolt of reality: It strongly looks to me like the OSM depth data project is a stupendous failure! I tried to say the same thing more politely in the entry, and that was before I realized that this project has been underway for two full years. Despite the good tools you built and the noble ambitions, hardly anyone is contributing data. How will you get more contributors when they realize how few boaters they’ll be sharing their data with, and how little data they’ll be getting back?
And, sorry, but it’s just sad to me to watch OSM and TeamSurv sniping at each other while they’re both being steamrolled by the big developers of commercial charts and marine electronics who actually understand that community sourcing takes a big community to be effective, and who have the resources to create those communities.
I believe that there are lots of boaters like me who would like to share their depth data as widely as possible. Hopefully, some entity will make that possible beyond any single community sourcing project.
Ben,
Yours is a great idea but, as I’ve said before, I don’t see how it is sustainable.
To collect data and make it freely available to Garmin, Navionics etc is bound to fail without a means of covering costs.
The biggest cost (whether financial or time) is in recruiting participants – as Navionics acknowledge in their plea to get data acquisition out of the hands of the hardware suppliers, who can build it in and distribute it at minimal additional cost.
Yes, there can be some revenue in having better algorithms and charging for data processing, but for many applications I don’t know if that would generate much revenue, nor how many would buy it (e.g. CloudMade as I referred to earlier).
As you yourself say, the value is in the data set, and if that is available free to all then how can the system be sustained?
Over to you!
Tim, I think we have to agree to disagree. I think that a fully open source depth database is sustainable. Even if the entity that runs it goes away (bankruptcy, loss of interest from maintainers…) the data is still there- under an open license. It can be picked up by anybody else, who then continues the project.
Well the bathymetry guys of the OpenSeaMap project apparently where not concerned with someone taking the data and not contributing back. Otherwise they would not have picked a public domain like license. If one would want to prevent such a thing one would pick a “copyleft” or “share alike” license. These prevent exactly the use case you are describing – like someone taking the data and not contributing the derived work back to the community. BTW, such a type of license worked very well for Wikipedia or the Openstreetmap project.
Simply preventing commercial use is IMHO too restrictive. What happens if someone wants to use the depth data on his commercial vessel?
Talking of the openstreetmap project. They might be doing it on a shoestring, but the outcome is phenomenal none the less IMHO.
Anyways, I wish you and the whole TeamSurv project all the best. If my boat would spit out GPS tracks with depth data, I’d happily upload them to all of the crowd- sourced bathymetry projects. Navionics, TeamSurv, and OpenSeaMap ;-).
Best Regards,
Christian
Ben – you’re being patently unfair to the Open SeaMap project. Where was Linux after in 1993 (2 years in)? It had a contributor community, but version 1.0 of the Linux kernel didn’t occur till 1994!
Now, we all know how well that open source project went.
As many proponents hold in this thread, there is no sustainable commercial entity that can maintain a pool of depth/bathymetry data. I wouldn’t expect them to either, they have a fiduciary duty to generate profit for owners.
The only viable options are either open source or government.
The train is moving slowly, but will gain momentum and will eventually be too fast for commercial closed and proprietary vendors to ignore/avoid. Just remember a few years ago about how few options existed for our plotters to display government charts (no you had to go buy proprietary chart cards based on OUR GOVERNMENT PROVIDED data???). Now, even Raymarine has gov’t charts (of sorts)..
Hi Ben,
the OpenSeaMap project is a somewhat heterogeneous group of volunteers that have one common goal- making an nautical map under a libre and open source license. I’d like to elaborate my point of view a bit more:
The license OpenSeaMap uses for the depth data is the ODC Public Domain Dedication and Licence [ http://opendatacommons.org/licenses/pddl/1-0/ ] guarantees exactly what you want.
(“I want to submit my data to a pool where it just sits untouched but available without constraint to any non-commercial or commercial developer that wants to use it. In effect, it would be held in a trusted pool but “owned” by all.”)
The data is free for the taking. One isn’t even obliged to contribute anything back to the community, not even a citation is necessary. I have to admit that as of now there is no mechanism in place to download a database dump, but this will come in the future. Give the developers time.
“How can we prevent, that commercial organisations take the data without contributing data?”
I also don’t understand this sentence from Markus. As I wrote earlier in response to Tims post there are licenses that prevent exactly that (copyleft, share alike licenses). One example of such a license is the Open Database License [ http://opendatacommons.org/licenses/odbl/summary/ ]. Apparently the initiators of the OpenSeaMap depth project were not concerned with this to be a problem, otherwise they would have chosen a different license.
The slow pace of the OpenSeaMap bathymetry project is indeed noticeable. If you compare it to – lets say the also crowd sourced Navionics Sonar Chart it doesn’t seem to anywhere.
Well, I read your article about the Navionics Sonar Chart on Garmin devices and all of the comments. I also took a good look at the pictures. While the tightly packed depth contour lines look impressive at first glance one really has to wonder how much is actually new and rich data and what is just interpolated by computers. To me it looked more like the latter.
Even Mr. Carnevali from Navionics wrote in the comments that the actual depth data is taken from a lot of sources, not just crowd sourced. If one sees how few tracks the Openseamap project got together in two years, one cannot imagine that the crowd contributed a substantial amount of data to the Sonar Charts.
Currently there is no comprehensive dataset available under a license that is compatible with OpenSeaMaps license. Hence Openseamap had to start from scratch. This is why it looks really scarce at the moment. Crowd sourced projects take time to gain momentum, especially when it comes to such a niche market like bathymetric data, but it is there, it won’t go away, and it can only get better and bigger with age. Will it prevail? I don’t know, but I surely hope so. If it catches on, it will be the freest depth dataset available. What’s wrong about that?
All that said, I am also a normal Yachtie and I am very happy about every additional bathymetric dataset that is available. I genuinely appreciate the efforts that projects like TeamSurv are putting into their work. If I came across as snippy then I really have to appologize because this was not my intention. I merely wanted to clarify the differences between an open source project like OpenSeaMap and e. g. TeamSurv.
Best Regards,
Christian
I noticed that “Olex users are encouraged to share their sea floor data” : http://www.olex.no/dybdekart_e.html
I wonder what’s in the logger box. It’s only 30 euros, so it can’t be too fancy. Maybe it’s a something like a “TP-Link TL-WR703N” w/ OpenWRT.
I’ve been trying to find inexpensive CANbus / NMEA-2000 adapters, but it’s difficult.
It’s not like RS-422 / NMEA-0183. I bought a USB to RS-422 adapter on Ebay the other day for $5. Works great.
The cheapest I’ve found so far for USB to CANbus is the “Leonardo CANbus” for 23 quid. I got it working with an automotive CANbus, but the firmware is “roll your own”, so I haven’t been able to get it working with OpenWRT.
The “8devices” gadget should work w/ OpenWRT (& gpsd), but it’s a bit pricey at $65.
There is this new “CANtact” gadget : http://cantact.io/ , but still not cheap at $60. It’s open source : https://github.com/CANtact . A nice enhancement would be to add some RS-422 transceivers so it could do NMEA-0183 as well. The firmware is “SLCAN”, which is a bit lame.
I’ve been playing with a “BeagleBone Black”, which runs Linux and has 2 CANbus controllers built in (as is common in ARM chips). There are no transceivers, so I have to add that.
Check out http://www.gpsbabel.org/. They have done a spectacular job writing software that converts various GPS file formats from one to another. It seems to be donation supported and GPL. OSM can probably learn a few things from them, and it seems like it would be useful for OSM to share any filetypes that get decrypted.
For me – I’d love some sort of Android/iPhone app that can log the NMEA data (although I’d need to connect it to the network somehow). Ultimately something that works seamlessly in the background is going to be the way to do this best.
Hey there,
So I stumbled across your article and have to say one thing.
YES FOR THE LOVE OF ALL THAT IS ALIVE YES!!!!!!!!
I find it absolutely infuriating as someone who is trying to find raw sonar data that I can’t access ANYTHING in Canada simply because everyone here has uploaded everything to Navionics, which won’t release ANY data or share any of it with anyone unless they pay through the nose.
Not to mention, the scientific community is lacking this data, and have an extremely hard time accessing anything of decent quality.
Hi,
Love your ideas and I think many would contribute if the manufacturers made it easy for us to extract files from the devices and give us the data in some sort of acknowledged file format in order for us to share it with likes of opens street charts.
Its now 2022., you wrote this 7 years ago, time for an update and push again maybe?