Raymarine and Mercury’s preview of semi-autonomous docking assist
For many of us docking maneuvers are the most stressful part of boating. So engine manufacturers, electronics companies and boat builders have introduced systems like integrated joystick control and dynamic positioning to make the task easier. Last Friday I got hand’s on with a new technology from Raymarine and Mercury Marine that takes docking assistance to a new level. They call it Virtual Bumper and I can attest that it lets you dock like a pro, even if you’re not.
With Virtual Bumper you still drive the boat, but 3D cameras are monitoring the situation in all directions and if any object violates a three-foot perimeter, the system takes over the controls and gets you back to safety. Once the boat is in a good docking position, you can simply turn the bumper off, or you can even tell it to bring the boat alongside the dock or seawall. I have lots of details to share about how it works and what it’s like to use.
What I saw installed on a Boston Whaler Outrage 330 at Mercury’s Lake X test site is a combination of Raymarine’s DockSense with Mercury’s Advanced Pilot Assist (APA) system. (APA is the umbrella term Mercury uses for their joystick piloting, autopilot, and Skyhook dynamic positioning.) One of the metaphors offered to explain the relationship between these two systems is that DockSense is the eyes and ears while APA is the hands and feet of the operation. DockSense builds a model of all the objects around the boat which it then shares with Mercury’s APA. APA then makes decisions about which objects have gotten or are getting too close and uses the outboards to maintain the 3-foot Virtual Bumper radius used by the system demonstrated Friday.
The 3-foot radius is fixed for now but will almost certainly be configurable on a released version of Virtual Bumper. Although Raymarine and Mercury both made it clear that what they demonstrated is not yet production-ready, it seemed polished and worked well. I never saw the system get confused or otherwise stumble while the corrections it made were smooth and proportional to the hazards.
Using the Virtual Bumper
As soon as the Virtual Bumper (represented as the blue glow around our boat on the DockSense screens) is activated, the vessel will move so the bumper zone is clear on all four sides. During our demonstration, the Whaler started off with the starboard side to a seawall. Once the Virtual Bumper was activated the boat moved to port to create the 3-foot safety margin. As we floated around another boat pulled within a few feet of our bow at which point the Outrage backed itself up a bit to preserve the safety margin (cool!).
We then moved away from the seawall and towards another seawall and floating dock, all under the watchful eye of the DockSense cameras. As we backed toward the floating dock we were on a line that would have brought our stern within 3 feet of the dock. Pulling straight back on the joystick resulted in the boat sliding straight back and then sliding just enough to port to ensure the three-foot margin was preserved. Mercury explained the new version of APA on our test vessel also corrects for environmental conditions, so if there were a strong current or wind pushing the boat sideways, pulling straight back on the joystick would still result in the vessel moving straight back as APA makes appropriate corrections.
The Virtual Bumper controller
The helmsperson interacts with the Virtual Bumper controller pictured about with its directional pad and blue glowing LED ring. When DockSense detects an object, the corresponding segment of the LED ring will blink. If you then tap that direction’s button the Virtual Bumper for that direction will be deactivated and the system will let you maneuver closer to an object on that side. If instead the operator presses and holds the arrow button the vessel will autonomously move alongside the object on that side and hold it there. Until one of the two docking modes is activated the system simply won’t let the vessel come within 3-feet of any object. If the joystick is moved towards an object the command isn’t allowed until the Virtual Bumper is deactivated.
How it works
-
- The hardtop of the Whaler Outrage 33 has three stereo cameras and the AHRS/GPS unit
-
- The front of one of the stereo cameras
-
- Two cameras on the bow of the Whaler, these cameras are mounted in the bow cleat position
-
- The aft facing camera mounted high on a mast to see over the outboards
DockSense is composed of five stereo vision cameras, each with two image sensors and an AHRS (attitude and heading reference system) built in, a sixth AHRS with GPS also built in on the hardtop, and the DockSense Processor (a black box that contains the considerable processing power required). The processor uses the various AHRS inputs to sense precise pitch and roll so it can correct for possible issues like the face of a wave or a small wake that the cameras might be seeing as a result of the boat’s movement.
This is a technology demonstration project so I’m hopeful production versions of the system will employ smaller, less obtrusive cameras and possibly fewer sensors. Though neither Raymarine nor Mercury discussed pricing, I suspect the current system with myriad sensors and lots of processing power is quite expensive.
Raymarine is using machine vision technology to map nearby objects. Images from the two cameras in each module are compared to compute the distance to an object, similar to what the human brain does as it interprets the difference between what is seen by the left and right eyes. The cameras and machine vision technology come from FLIR (which recently acquired specialist Point Grey Research), but Raymarine developed the marinized housing with the built-in AHRS and a powered shutter to prevent spray from getting on the lenses underway.
-
- Our test well for the day
-
- DockSense’s plotting of our position in the well
In the pictures above you can see the actual location of the Outrage and the visual model DockSense built using the output of the five stereo vision cameras and which typically covers a 50 square meter area centered on the boat. Note that there are three distinct color codes: gray indicates an object previously mapped but not currently seen, green indicates a higher object like the seawall (in this case) currently seen, and light blue shows lower objects like the floating dock, also currently visible to the system.
The demonstration Boston Whaler has excellent sight lines but the DockSense display on the Raymarine Axiom MFDs is good enough that I’m pretty confident it would work well with much poorer visibility of nearby objects. In fact, I think that the DockSense visualization technology would be a real benefit at the helm without the Virtual Bumper propulsion control.
I’ll also note that because the DockSense cameras can measure distances to an object, they’re not fooled by something like a crew person walking in front of a camera. If something is detected within the perimeter of the boat, it’s simply ignored.
The technology demonstrated at Lake X feels like the very tip of the iceberg of autonomous boating, though I think most recreational boaters are likely to be more interested in docking assist technologies like Virtual Bumper rather than fully autonomous operations. But then again, Volvo Penta has already demonstrated a fully autonomous self-docking system aboard an IPS powered boat, though that system requires beacons placed on the dock.
Raymarine has also partnered with Prestige to integrate DockSense onto an IPS powered Prestige 460 to deliver similar Virtual Bumper capabilities. They’ve made a pretty entertaining video touting the system’s ability to make anyone look like a pro.
Adding Virtual Bumper to the numerous other assistive technologies already on the demo boat make it seem as though more ambitious capabilities aren’t far away. In fact, the current capabilities of DockSense seem a natural fit for underway collision avoidance. I should also mention that I demoed the technology on a nice day during full daylight without inclement weather, significant wind or current. I can’t say how the system would have performed in more adverse conditions but I can say that it performed without a hitch in good conditions.
Looks cool – for the sort of vessel that can accomplish motion in all directions – but no mount of sensors or smarts is gonna move Atsa backwards downwind 🙂 When the smart steering gets good enough to do the prior planning and setup work required to get us into a crosswind slip, I’ll be very interested!
Hartley, these systems are all based on the latest and greatest technology and take advantage of all the propulsion controls available. It’s safe to assume that we will see the capabilities roll out on the systems with the most advantages first. We will probably see some options become available for older propulsion systems but I’m not sure we will ever see options for a single screw, no thruster sailboat or trawler. I’d guess it’s more likely we will see pod type sail drives before we retrofits are made for fully mechanical propulsion systems. Part of the beauty of what’s being done here (and the curse when things go wrong) is that both the DockSense & Mercury system and the DockSense & Prestige systems all link at the control systems, not at steering or throttle linkages.
Hi Ben, Oh, I know that 🙂 But I do enjoy being a curmudgeon sometimes… And we DO have a thruster now, so my life is significantly more serene. Are you in Florida now? We are still here in Riviera Beach, probably thru early March.
Hartley, I figured you knew but also wanted to make sure everyone understands the complexities.
We’re in the St. Petersburg area for another month or so before we head down to Fort Meyers for around a month.
Great – just what we need, another crutch for those too lazy or clueless to learn the basics of boat handling, let alone actually take pride in mastering a skill. This reminds me of the millennial who panicked because the backup camera on his minivan fogged over and he couldn’t figure out how to use the mirrors. Someone needs to remind these engineering geniuses that just because something is technologically possible does not mean it’s a positive influence in the real world, where most of us do our boating. I see a future where one blown fuse, and we have a skipper who’s not just a little rusty, but completely incapable of controlling his boat. And as far as anti-collision goes, seriously? ANOTHER excuse to encourage folks to be heads down, trying to find their favorite playlist on the MFD, while assuming the system will save them from running into they guy ahead of them?
Let’s leave this trend to the automotive world, where we long ago gave up the premise of folks actually mastering any skill or even putting their phones down. We already have enough rookies grinding away on their bow/stern thrusters in perfectly flat and calm conditions, trying to put a 40’ boat in a 60’ slip. Why encourage more incompetence?
And for goodness sake, Mercury, could you not even figure out a more appropriate name?? As I often need to remind my landlubber friends, “It’s actually a FENDER, not a bumper.” Good grief.
[end rant]
Grant,
Your rant makes me a little uncomfortable. Boating’s demographics are skewing older and our already a small market risks getting even smaller.
Many technology advances are met with consternation about the dumbing down of the underlying activity. Though I take pride in my boat handling abilities (and hence take a mental hit when I do something stupid) I won’t judge those who are learning to boat and make use of aids. It’s possible to navigate across oceans with nothing more than a compass and a chip log but that doesn’t make it the best way to do that.
The only way those rookies grinding away on their thrusters will get more skilled is to get out there and use their boats. If this or similar systems make them more comfortable doing that I’m at a loss to figure out why that’s a bad thing.
Commercial airplanes with two highly skilled pilots at the controls have collision avoidance systems, if such a system can avoid a collision why is that a bad thing?
As for the name… I agree entirely and cringe when I see it.
Ben S.
Ben – sorry for the rant – kind of. Not sure I buy your arguments though. I agree with your comments on the market demographics – but is it worth having a bunch of buffoons out there with no idea what they’re doing, just so manufacturers can sell more boats? That’s not really my concern. I suppose you could make an argument that we all benefit from a bigger market place and more technology, but when is it enough? How much more technology do we need to enjoy a day on the lake or a weekend cruise? Answer: Zero.
You say you wont judge those who are “learning to boat and make use of aids”. But that’s the point – they’re NOT learning to boat, they’re learning to fool around with a joystick like a video game. Maybe it should be a Geico commercial – “So easy, a caveman could do it”. Sure, high end yachts might use this technology routinely – but they’re being skippered by experienced captains who can take over manually in a flash with no issues.
It’s also not true that the “only way those rookies grinding away on their thrusters will get more skilled is to get out there and use their boats”. The alternative is quite simply to put the time and money towards on-water training from a qualified instructor. But that requires some effort and commitment, and a seriousness of purpose. Also accepting the responsibility of sharing the waterways with everyone else.
Finally – your analogy about commercial airlines – yes, they have collision avoidance (TCAS). You know what drove the development? A couple of tragic mid-air collisions between airliners and private pilots who blundered into the wrong airspace. And even that system and protocol doesn’t involve using autoflight systems to maneuver the aircraft – it’s a visual/aural alert that requires the pilots to DIS-connect the autopilot and hand-fly an avoidance maneuver….. that whole “skill thing” again…
Hi Grant,
First, I’ll concede my “curmudgeon” title to you 🙂
Second, how many boaters NEVER move their boats because doing so is too nerve-wracking? I bet the number is pretty high. I know our willingness to maneuver in & out of marinas and other tangly spots has risen significantly since we installed a bow thruster. Now a bow thruster isn’t “joystick” maneuvering, but it sure makes things safer and easier on our single-screw sailboat. Did we maneuver in & out without one? – sure – but there were times when things got pretty hairy. Just because we could do without for a “weekend cruise” doesn’t mean it isn’t better having the advanced tools aboard and available.
I do think that if you want to challenge yourself with technology from a century or more ago, you have my full blessing – but please be VERY CAREFUL docking your barky next to me when the wind pipes up and the current is running. 🙂 Just because someone has more advanced tools doesn’t make them a buffoon – that’s a separate skill set entirely, and the tech on the buffoon boat probably won’t make much difference to the outcome.
Fourth, I might also point out that the “professional mariner” world is going to “joystick” maneuvering in a big way – and for good reason, since even a minor crunch can be mighty expensive in both time & money. Having these controls means they can get the job done faster and safer than they could without them – and I think the same applies to us recreational mariners as well.
Fair Winds!
Hello Hartley – title transference accepted. 😉 Although I may only be able to fill in part time…
I totally agree that a bow thruster, or even a stern thruster is completely appropriate for some vessels – a single screw trawler or sailboat like your is a perfect example. And I would never fault anyone for using one as necessary. What I referred to is the opposite – totally UN-necessary use of thrusters on twin-engine cruisers in still, quiet and calm conditions, simply because the “helmsman” has never bothered to learn basic boat handling skills. I’m sure there ARE boaters who never move their boats, because it’s too nerve-racking for them. I don’t have a lot of sympathy, any more than I would for someone who buys an airplane and then was afraid to fly it. Training is readily available ( no, I’m not selling it) and ought to be a no-brainer if you have the slightest concern for your prized new boat and the passengers in your care.
I have no issue with anyone adding/using whatever technological bells and whistles they choose, and I’m not advocating we go back to technology from a century ago. I’m saying that basic boat handling should b e mastered BEFORE resorting to automation, not discarded entirely in favor of it – which is precisely what I see happening. If your GPS fails – can you hold a compass heading? If your Cell phone dies, can you operate a VHF properly? And if your auto-bumper, or whatever the hell its called fails – and it will, at the worst possible time (See Murphy) – can you safely guide your boat to a dock without hitting anything?
If the answer is no, then you probably ought to be spending your time and money learning a few things, rather than buying more gadgets.
Finally in regards to your professional mariner comment – I understand that, and I understand why. But these guys have nothing to prove. Technology is a workload reducer for them, where fatigue and weather are daily challenges – not a crutch because they never learned seamanship in the first place.
Fair winds to you as well,
Really cool! Does it also work in fogs where the visibility is low ?
Raymarine has mentioned that, at least this technology demonstration, is impacted by rain. This is technology is using machine vision to measure and understand the world around the boat. If the machine can’t see, it’s going to have troubles. I’d expect that much like a person, the worse the conditions the harder time DockSense will have. I think it’s likely subsequent versions will have other sensors (ultrasonic, RF, IR, etc) to help with these conditions.