NASA field campaigns – which study Earth from air, sea, and on the ground – return vast troves of valuable data. Sometimes, however, processing so much raw information and related satellite imagery can slow the pace of scientific discovery.
To combat that lag in output, researchers and programmers have developed a prototype 3D video dashboard for data from field campaigns that will make information easier to process and more readily available to anyone across the Earth science community.
“Navigating and visualizing thousands of raw data files collected during a field campaign is an extraordinarily tedious task, and – left as is – might reduce the scientific return of the campaign,” said Larry Di Girolamo, a professor of atmospheric sciences at the University of Illinois.
Di Girolamo collaborated with programmers and designers from the Advanced Visualization Lab (AVL) at the University of Illinois Urbana-Champaign National Center for Supercomputing Applications in Urbana, Illinois, to prototype a new process for communicating data from field expeditions.
The final visualization prototypes include a three-minute video in 3D, as well as a longer data dashboard video depicting 76 different data variables. These visualizations – as well as the raw data from the CAMP2Ex field campaign – meet NASA’s Open Source Science requirements and are freely available to the public.
The project was supported by the Advanced Information Systems Technology (AIST) program in NASA’s Earth Science Technology Office (ESTO).
The process brought together a combination of commercially available, open source, and home-grown software to transform raw data from NASA’s recent Cloud, Aerosol and Monsoon Processes Philippines Experiment (CAMP2Ex), into a 3D animation and a data dashboard video. CAMP2Ex explored the impact air pollution may have on weather patterns in the Philippines, and these visualizations made it easier for researchers to locate patterns within the thousands of data points collected during the field campaign.
The Camp2EX campaign data dashboard video is the first to incorporate such a dynamic array of data sources for a NASA field expedition.
In the Philippines, high-altitude clouds can impede satellite observations of the interaction between lower-altitude clouds and the airborne particles, such as pollution, that affect rainfall and the weather. To fill the gap, additional data for Camp2EX was collected by Research planes, a U.S. Navy research vessel, and the Manila Observatory in Quezon City.
Using these myriad instruments to compile a robust record gave scientists the complete data picture they needed to conduct their research, but also left them with the challenge of finding patterns within so many different data points.
“How do you build a comprehensive picture from hundreds of thousands of files from different data sources?” asked Di Girolamo.
To solve this problem, Di Girolamo and the creative team at AVL prototyped two visualization products using data from CAMP2EX’s Research Flight 09.
The first visualization was a 3D animated video that depicts the journey of NASA’s P3 research aircraft as it navigates the open ocean south of Luzon. The second was a prototype of a potential interactive dashboard that presents data gathered by airborne, seaborne, and spaceborne instruments supporting that research flight.
Credits: NASA / AVL
The CAMP2Ex exhibition video, illustrating data gathered during CAMP2Ex’s Research Flight 09. During this flight, Di Girolamo’s team gathered data linking high concentration of aerosols near the Philippines to biomass burning in Indonesia.
“In one place, you have all the information you need to locate patterns between these different data sets,” said Stuart Levy, a senior programmer and system administrator at AVL. “As the video plays, the data in the dashboard changes in real time, so you can see exactly when and where a certain data point appeared in the record.”
“These visualizations could revolutionize how we communicate our data to other researchers and members of the public. It could make field data easier to understand for anyone interested in fundamental research,” said Di Girolamo.
Kalina Borkiewicz, a senior research programmer at AVL, noted that, “The scope of this project made it unique. We had to work with data from numerous instruments in different spatial scales, temporal scales and data formats. Making sure the final product was seamless took a lot of work.”
The visualizers used the Blue Waters Supercomputer at the National Center for Supercomputing Applications to generate 216,000 individual images and more than 1.5 million intermediate images, which they stitched together into a product package that organized the CAMP2Ex data visually across space and time.
“We’re excited for the possibility of organizing our other datasets like this and sharing this process with other researchers handling Earth-science data,” said Di Girolamo.
NASA’s Advanced Information Systems Technology (AIST) program identifies, develops, and supports novel software and information systems like the CAMP2Ex visualization process that will help Earth scientists learn more about the world we live in.
NASA 实地活动——从空中、海洋和地面上研究地球——返回了大量有价值的数据。然而，有时处理如此多的原始信息和相关的卫星图像会减慢科学发现的步伐。
为了解决输出滞后问题，研究人员和程序员开发了一个原型 3D 视频仪表板，用于存储来自野外活动的数据，这将使信息更易于处理，并且更容易为地球科学界的任何人提供。
“导航和可视化在实地活动期间收集的数千个原始数据文件是一项非常乏味的任务，并且 - 保持原样 - 可能会减少活动的科学回报，”拉里迪吉罗拉莫说，大学大气科学教授伊利诺伊州。
Di Girolamo 与位于伊利诺伊州厄巴纳的伊利诺伊大学厄巴纳-香槟分校国家超级计算应用中心的高级可视化实验室 (AVL) 的程序员和设计师合作，设计了一种新流程的原型，用于从实地考察中传递数据。
最终的可视化原型包括一个三分钟的 3D 视频，以及一个描述 76 个不同数据变量的更长的数据仪表板视频。这些可视化——以及来自 CAMP2Ex 实地活动的原始数据——符合 NASA 的开源科学要求，并且可以免费向公众开放。
该项目得到了 NASA 地球科学技术办公室 (ESTO) 的高级信息系统技术 (AIST) 计划的支持。
该过程结合了商用、开源和本土软件，将来自 NASA 最近的云、气溶胶和季风过程菲律宾实验 (CAMP2Ex) 的原始数据转换为 3D 动画和数据仪表板视频。 CAMP2Ex 探索了空气污染可能对菲律宾天气模式的影响，这些可视化使研究人员更容易在实地活动期间收集的数千个数据点中定位模式。
Camp2EX 活动数据仪表板视频是第一个为 NASA 实地考察包含此类动态数据源阵列的视频。
在菲律宾，高空云可能会阻碍卫星观测低空云与影响降雨和天气的空气颗粒物（如污染）之间的相互作用。为了填补这一空白，Camp2EX 的额外数据由研究飞机、美国海军研究船和奎松市的马尼拉天文台收集。
使用这些无数仪器来编制可靠的记录为科学家们提供了进行研究所需的完整数据图，但也给他们带来了在如此多的不同数据点中寻找模式的挑战。
“你如何从来自不同数据源的数十万个文件中构建一个全面的图景？”迪吉罗拉莫问道。
为了解决这个问题，Di Girolamo 和 AVL 的创意团队使用来自 CAMP2EX 的 Research Flight 09 的数据制作了两个可视化产品的原型。
第一个可视化是一段 3D 动画视频，描述了 NASA 的 P3 研究飞机在吕宋岛以南的公海航行时的旅程。第二个是潜在交互式仪表板的原型，该仪表板展示了支持该研究飞行的机载、海上和太空仪器收集的数据。
显示研究飞机的图形数据集、飞行轨迹和窗外视频。它们一起显示了同时获取的不同数据。
CAMP2Ex 展览视频展示了 CAMP2Ex 09 号研究飞行期间收集的数据。在这次飞行中，Di Girolamo 的团队收集了将菲律宾附近高浓度气溶胶与印度尼西亚生物质燃烧联系起来的数据。
AVL 的高级程序员和系统管理员 Stuart Levy 说：“在一个地方，您拥有在这些不同数据集之间定位模式所需的所有信息。” “随着视频的播放，仪表板中的数据会实时变化，因此您可以准确地看到某个数据点出现在记录中的时间和位置。”
“这些可视化可以彻底改变我们将数据传达给其他研究人员和公众的方式。它可以让任何对基础研究感兴趣的人更容易理解现场数据，”Di Girolamo 说。
,
