Pierluigi Paganini November 18, 2017

Three AWS S3 buckets containing dozen of terabytes resulting from surveillance on US social media were left wide open online.

It has happened again, other three AWS S3 buckets containing dozen of terabytes resulting from surveillance on US social media were left wide open online.

The misconfigured AWS S3 buckets contain social media posts and similar pages that were scraped from around the world by the US military to identify and profile persons of interest.

The huge trove of documents was discovered by the popular data breach hunter Chris Vickery, the three buckets were named centcom-backup, centcom-archive, and pacom-archive.

CENTCOM is the abbreviation for the US Central Command, the US Military command that covers the Middle East, North Africa and Central Asia, similarly PACOM is the for US Pacific Command that covers the Southern Asia, China and Australasia.

Vickery was conducting an ordinary scan for the word “COM” in publicly accessible S3 buckets when spotted the unsecured buckets, one of them contained 1.8 billion social media posts resulting from automatic craping activities conducted over the past eight years up to today. According to Vickery, it mainly contains postings made in central Asia, in many cases comments made by  US individuals.

Documents reveal that the archive was collected as part of the US government’s Outpost program, which is a social media monitoring and narrowcasting campaign designed to target youngsters and educate them to  despise the terrorism.

The archive discovered by Vickery in fact includes the Outpost development, configuration files, as well as Apache Lucene indexes of keywords designed to be used with the open-source search engine Elasticsearch.

“While public information about this firm is scant, an internet search reveals multiple individuals who worked for VendorX describing work building Outpost for CENTCOM and the Defense Department” reads the blog post published by Upguard.

Another folder titled “Coral” likely refers to the US Army’s “Coral Reef” intelligence software.

“This folder contains a directory named “INGEST” that contained all the posts scraped and held in the “centcom-backup” bucket. The Coral Reef program “allows users of intelligence to better understand relationships between persons of interest” as a component of the Distributed Common Ground System-Army (DCGS-A) intelligence suite, “the Army’s primary system for the posting of data, processing of information, and dissemination to all components and echelons of intelligence, surveillance and reconnaissance information about the threats, weather, and terrain” programs. Such a focus on gathering intelligence about “persons of interest” would be even more clear-cut in the other two buckets, starting with “centcom-archive.” continues the post.

The bucket “centcom-archive” contains an impressive volume of internet posts stored in the same XML text file format as seen in “centcom-backup,”  at least 1.8 billion such posts are stored here.

“The bucket “centcom-archive” contains more scraped internet posts stored in the same XML text file format as seen in “centcom-backup,” only on a much larger scale: conservatively, at least 1.8 billion such posts are stored here.” states the post.

It is disturbing how this material was leaked online due to misconfigured AWS S3 buckets, foreign governments and terrorist organization may have had access to the archive such as Vickery.

Vickery notified the American military about the discovery and the buckets have now been locked down and hidden.

It isn’t the first time that data from US Military was discovered online, in September researchers from cybersecurity firmUpGuard discovered thousands of files containing personal data on former US military, intelligence, and government workers have allegedly been exposed online for months.

Pierluigi Paganini 

(Security Affairs – AWS S3 buckets, surveillance)

[adrotate banner=”5″]

[adrotate banner=”13″]