COTS Logo


UAV Data Imaging Solutions Push Limits of Embedded Technologies

The evolution of UAV-based image collection has a long mission-critical history for the U.S, military. Today’s hi res image capture technology requires 6,000 Terabyte per flight performance.

David Lippincott, Chief Technology Officer Chassis Plans | April 2016

Aerial surveillance systems mounted on UAVs generate huge amounts of imagery data, up to 6,000 Terabytes/flight per UAV. This data is more often than not created in remote global locations with poor or non-existent pathways for transmitting that data to the people that need it. One such system is the U.S. Air Force Gorgon Stare which utilizes a 1.8-billion pixel camera system to image a 50 square kilometer circle (Figure 1). Modern sensor syAMP RTC COTS body ad - April 2016stems used in persistent surveillance platforms can produce more than 100 Gbytes/s of data when tasked with wide area surveillance purposes. This article discusses the Gorgon Stare program, data management and a custom transit case data system manufactured by Chassis Plans that is used to securely transport that reconnaissance data.  

Identifying enemy threats has always been a prime doctrine of any battle. The more so in the current battle space in Afghanistan. When your troops are surrounded by buildings in areas crowded with civilians, how do you keep track of the enemy who mean your troops harm? How do you track enemy combatants traveling in remote areas without a massive deployment of friendly troops? To this end, the military has been searching for the high ground in any battle environment; first hill tops, then balloons and now UAVs and aerostats.

Bandwidth has always been a problem; too much information and no way to efficiently transmit that information in real time to the battle commanders who need it. The advent of radio opened up more remote reconnaissance allowing real-time reporting but it was still word of mouth. Television after WW2 allowed true real-time remote surveillance but was still limited in terms of the area and resolution that could be observed.

FINAL COTS04 Chassis Plans Fig1_original

Figure 1.

MQ-9 Reaper showing Gorgon Stare ARGUS-IS installation.

The Age of the Predator

The Predator UAV was developed in 1994 with the first flight on July 3, 1994. Prior to the Predator, UAVs had been small, lightweight and unobtrusive drones. As such, they had very limited payload capability, perhaps offering a film camera or a low-resolution video image transmitted line-of-sight to their controller to help determine enemy disposition. Predators, on the other hand, are big with a wing span of 49 feet and an empty weight of 1,100 pounds allowing a payload of approximately 500 pounds. Predators are controlled remotely via satellite and can operate anywhere in the world with the operators being located in the U.S.A. Image data can be uplinked via satellite or by line-of-site to local commanders.

This payload capability of the Predator, much more than offered by small UAVs then in use, allowed the installation of sophisticated sensor suites including stabilized visual and infrared camera balls that transmitted images in real-time to ground stations for analysis and dissemination to troop commanders. Finally, the military had high-resolution eyes on the high ground, safe from enemy ground fire, mobile with long loiter times and able to dispatch to any part of the battle space, providing real-time reconnaissance night and day. These aircraft could be flown into areas denied to traditional manned aircraft as potentially too dangerous.

However, the early cameras on the Predator had a limited field of view and were likened by pilots to “flying the aircraft while looking through a straw.” They were a huge improvement over existing technology and were first deployed in Bosnia in 1995, providing valuable real-time intelligence. But the military wanted more. The Predator B, designated the MQ-9 Reaper by the U.S. Air Force, is a follow-on to the original Predator, is bigger with a 55 foot wing span, faster, offers longer loiter time and many improvements including a 3,850 pound payload capacity that includes 3,000 pounds of external stores.

But the sensor ball under the nose still suffered from a limited field of view, like “looking through a straw” as many operators have said. Operators could follow a single target, but multiple targets presented significant problems in tracking. The installed AN/AAS-52 multi-spectral camera system offers six fields of view, ranging between 19mm to 560mm. Thus, operators can zoom out for a wide angle view or zoom in to track a single target. The ball must be rotated to change the central target focus. The wide angle view showed a large area but lacked resolution.

 

Gorgon Stare

The Global War on Terror dramatically changed the way the U.S. Military used persistent surveillance to pursue terrorists and enemy combatants. The limited field of view of existing aerial platforms was a hindrance in keeping track of targets. It worked great for single or stationary targets but if the enemy scattered in different directions, it was difficult to reacquire the targets. Sierra Nevada Corporation was contracted to create a new high resolution camera which the U.S. Air Force planned to deploy in 2011 with a follow-on in 2012 and a third in 2014. This was Gorgon Stare Wide Area Airborne Surveillance System (WAASS) and was conceived, designed and developed in less than three years (Figure 2).

FINAL COTS04 Chassis Plans Fig2_original

Figure 2.

Predator versus Gorgon Stare surveillance area.

 

As initially released, Gorgon Stare was housed in two 500 pound pods which were mounted on the inboard weapons pylons on the MQ-9 Reaper. The right-side pod housed the sensor ball, manufactured by ITT Defense, and an image processor. The left side pod contained a computer to process and store the images, data-link modem, two pairs of Common Data Link and Tactical Common Data Link antennas plus radios.

The sensor ball contained five electro-optical (EO) cameras for daytime and four infrared (IR) for nighttime and poor visibility due to dust and smoke. These cameras were positioned at different angles to maximize ground coverage. The five EO cameras shot two 16-megapixel frames/sec each for an equivalent 80-megapixel sensor. The four IR cameras imaged 8-megapixels each for an equivalent 32-megapixel sensor. This was not true video which is normally 30 frames/s but was still better than fixed images. Data was captured at 15.3 Gbps. All video data was stored on-board for downloading after the Reaper landed for additional analysis.

 

Coordinated Operation

Gorgon Stare was operated independently, but in coordination, from the UAV pilots by two operators in a dedicated ground station in the battle theater. A separate processing, exploitation, and dissemination team co-located with the Gorgon Stare ground station coordinated with field commanders to get the images to the field. The data stream could be divided into 10 individual views allowing tracking of 10 targets and streamed to that many recipients via the Tactical Common Data Link. This was an order-of-magnitude improvement over the single camera carried in the nose-ball of the UAV.

Gorgon Stare would process the images from all the on-board cameras into a single mosaic for a wide-angle view which could be streamed to tactical operations centers. This relatively low-resolution image could provide an overview of the battle space. Instead of looking at a single building or vehicle, this image allowed examining an entire village. The mission data is stored on-board for post-flight download and analysis. Said Lt. Gen. David Deptula, deputy chief of staff for intelligence, surveillance and reconnaissance, “You can review [the data] and accomplish forensic study of the area by looking at movement and tracing activity. If you know where an improvised explosive device went off, you can ‘rewind the tapes’ and see where the activity was and what led to it.”  Gorgon Stare was very much better than the existing systems to that time but the U.S. Military wanted more.

 

ARGUS-IS Goes Beyond

Wanting more capability, the U.S. Air Force commissioned Gorgon Stare Increment II with over twice the coverage area and much higher resolution. ARGUS-IS, or the Autonomous Real-Time Ground Ubiquitous Surveillance Imaging System, was a Defense Advanced Research Project Agency (DARPA) program contracted to BAE Systems.

ARGUS-IS melded 368 5-megapixel cell phone cameras and four image-stabilized lenses to produce a single mosaic image with an effective resolution of 1.8 billion pixels (Figure 3). Compare that to your 20 megapixel Canon DLSR. While the official coverage area is classified, it is reported to be 35 to 50 square kilometers with the Reaper orbiting at 20,000 to 25,000 feet and an endurance of 12 to 14 hours. Two aircraft can provide 24 hour coverage. ARGUS-IS can resolve a six inch object while orbiting at 20,000 feet. Data is recorded at 15 frames per second as compared to 2 frames per second for Increment I and is detailed enough to resolve flying birds. The images are now in color versus the original black and white (grey scale) images.

FINAL COTS04 Chassis Plans Fig3_original

Figure 3.

1.8 billion pixel ARGUS-IS image of Quantico, VA.

.

 

ARGUS-IS is a two ball system with one ball providing daylight visual data (EO) and the other providing infrared images (IR). The balls are both gyro stabilized and mounted in separate pods. These balls are controlled by the Gorgon Stare ground station and the Reaper pilots retain control over the nose ball in the aircraft. On-board processing combines the data from the two balls into a single high-resolution mosaic image. Future enhancements include incorporating synthetic radar data for use in cloudy or dusty conditions. The Reaper has the payload capacity for additional equipment.

Of particular note is the system has the capability to stream 65 separate high-resolution moving images to that many users in real-time while the aircraft is in flight. Compare that to the original Predator with the nose ball able to watch only one target at a time and Increment I Gorgon Stare providing 10 image streams. In addition to raw image data, ARGUS-IS can provide latitude and longitude meta data for each pixel in the image.

 

Data Overload

Big data is good but the payoff is the actionable intelligence buried in the data. Analyzing the raw data to tease out the important information is as important as having the raw image data. High frame rate video streams and imagery from multiple sources in real time are be

coming more common, making the requirements for high-performance data transfer and storage even more challenging.

The ARGUS-IS camera provides a resolution of 1.8 billion pixels. 1.8 billion pixels at 12 frames per second generates on the order of 600 gigabytes per second or 6,000 terabytes of video data per day. That is a stack of 1.44Meg floppy disks 44,000 feet high. Massive data crunching onboard the Gorgon Stare platform reduces the raw data to something significantly more manageable but this is still a lot of data. The system will stream an overview mosaic and 65 small high-resolution pieces of the image mosaic in real time but there is not enough bandwidth to transmit the entire 1.8 billion pixel image at full resolution while the Reaper is in flight. On-board storage takes up this slack by allowing the entire flight’s data to be downloaded once the Reaper is back at base.

Processing takes place at multiple locations where raw flight data is ingested and remains at the edge (battle space) for tactical use while some data is distributed for broader analytical use. With the entire flight’s data available for post-flight analysis in conjunction with prior flight’s data, even from other Reapers, a long-term history of the area can be compiled allowing analysts to build a profile of day-to-day life looking for anomalies that portend terrorist activity. They have a time machine allowing them to go back in time to follow the path of a vehicle or person to an origin. This is the valuable information in the raw data.

The other problem is after the flight, the aircraft will be parked on the ramp while being fueled and preflighted for the next mission and is not necessarily near where the mission data is needed. Wi-Fi is out of the question and not nearly fast enough. Even at Gbit Ethernet speeds, downloading the data via a Cat 6 Ethernet cable, should one be available and able to survive the flight line environment, is not fast enough to download that much data in a reasonable time frame.

 

Transportable Solution

Working closely with the customer, Chassis Plans provided a very complex custom solution to provide the ground crew a rugged portable intelligent data transportation system. The system, easily carried by one person, is carried to the aircraft and used to download the flight data. High-speed communications to the aircraft is provided by a 10 gigabit fiber link. Sufficient high-speed RAID disk storage with in-system intelligence for management allows an entire flight’s image data to be stored in the system. The transit case system has to be sealed and rugged enough to survive flight line conditions in Afghanistan which can be very hot, very cold, very dusty and wet. Chassis Plans packaged the electronics and disc storage in a small transit case (Figure 4). The small size makes this raw post-flight data easily transported to the central command or even back to the United States for inclusion in the overall theater database for near real-time analysis using beefy super computers.

FINAL COTS04 Chassis Plans Fig4_original

Figure 4.

Chassis Plans Custom ARGUS-IS Data Transportation System.

Chassis Plans
San Diego, CA
(858) 571-4330
www.chassis-plans.com