FlightGoggles is envisioned to be development environment that allows the design, implementation, testing and validation of autonomous super-vehicles. FlightGoggles currently provides exteroceptive sensor simulation based on the Unity3D engine as well as vehicle dynamics and inertial sensor simulation capabilities. FlightGoggles is powered by photogrammetry - a technique that creates virtual environments through their photographs. To create both of the two FlightGoggles virtual environments, we took countless photographs of real-world assets. We created 3D graphics assets from these photographs. FlightGoggles provides advanced simulation capabilities, like virtual-reality and augmented-reality simulations.
The Stata Center Virtual Environment is a virtual replica of two floors of MIT's Stata Center - a Frank-Gehry-designed building. Stata Center is one of the landmarks of Boston area. It is the home to our parent laboratory, the Laboratory for Information and Decision Systems (LIDS). The Stata Center environment is carefully crafted from tens of thousands of photographs. The environment is hand-cleaned in order to perform rendering in real time. The Stata Center environment is truly unique as being one of the most visually-compelling virtual replicas available today. In most places, it is an excellent match for the real Stata Center. A formal comparison is difficult to make. But, we provide some examples of visual benchmarks. More information and visuals are here.
The Abondoned Factory Virtual Environment is created by creating virtual replicas of eighty different objects, such as columns, walls, rubble, barrels, and more. The environment is then put together by creating around a thousand instances of these eighty different objects, hence replicating some of the objects many times. The environment features visually-challenging elements, such as water puddles, rubble, broken objects, and various lighting effects. The lighting can be modified to reflect different times of day. More information and visuals are here.
FlightGoggles Arcade is a drone racing video game, developed for research purposes. Specifically, our aim is to capture how human pilots make their decisions in drone racing games. It features FlightGoggles Stata Center Environment, which is a digital replica of MIT's Stata Center. The game has a number of courses with varying challenge levels.
FlightGoggles Arcade will be released for Mac and Windows shortly. Please stay tuned!
The video below is a teaser, showing the main menu and game play.
FlightGoggles was utilized as the simulation challenge to select nine teams for the AlphaPilot program. The videos below show the results from top twenty teams that participated in the challenge. Detailed results can be found in our recent research paper in the research page.
The teams were tested in 25 instances of the same environment. In each instance, the gate locations were slightly perturbed, and the perturbations were unknown to the contestants' algorithms.
All AlphaPilot related videos can be found in this Youtube playlist. The videos are listed one by one below.
This video showcases the results of the simulation challenge.
This video is an overhead of view of the challenge
This video shows a tiled view of front camera images streaming from all of the drones. The second video is no different, except we also added the trails for you to easily understand how many drones are ahead.
The videos below show the course from various static cameras. We added the trails to all of the drones so that you can see where each of them is.
