• Home
  • Cyber Crime
  • Cyber warfare
  • APT
  • Data Breach
  • Deep Web
  • Digital ID
  • Hacking
  • Hacktivism
  • Intelligence
  • Internet of Things
  • Laws and regulations
  • Malware
  • Mobile
  • Reports
  • Security
  • Social Networks
  • Terrorism
  • ICS-SCADA
  • POLICIES
  • Contact me
MUST READ

UK NCA arrested four people over M&S, Co-op cyberattacks

 | 

PerfektBlue Bluetooth attack allows hacking infotainment systems of Mercedes, Volkswagen, and Skoda

 | 

Qantas data breach impacted 5.7 million individuals

 | 

DoNot APT is expanding scope targeting European foreign ministries

 | 

Nippon Steel Solutions suffered a data breach following a zero-day attack

 | 

Iranian group Pay2Key.I2P ramps Up ransomware attacks against Israel and US with incentives for affiliates

 | 

Hackers weaponize Shellter red teaming tool to spread infostealers

 | 

Microsoft Patch Tuesday security updates for July 2025 fixed a zero-day

 | 

Italian police arrested a Chinese national suspected of cyberespionage on a U.S. warrant

 | 

U.S. CISA adds MRLG, PHPMailer, Rails Ruby on Rails, and Synacor Zimbra Collaboration Suite flaws to its Known Exploited Vulnerabilities catalog

 | 

IT Worker arrested for selling access in $100M PIX cyber heist

 | 

New Batavia spyware targets Russian industrial enterprises

 | 

Taiwan flags security risks in popular Chinese apps after official probe

 | 

U.S. CISA adds Google Chromium V8 flaw to its Known Exploited Vulnerabilities catalog

 | 

Hunters International ransomware gang shuts down and offers free decryption keys to all victims

 | 

SECURITY AFFAIRS MALWARE NEWSLETTER ROUND 52

 | 

Security Affairs newsletter Round 531 by Pierluigi Paganini – INTERNATIONAL EDITION

 | 

North Korea-linked threat actors spread macOS NimDoor malware via fake Zoom updates

 | 

Critical Sudo bugs expose major Linux distros to local Root exploits

 | 

Google fined $314M for misusing idle Android users' data

 | 
  • Home
  • Cyber Crime
  • Cyber warfare
  • APT
  • Data Breach
  • Deep Web
  • Digital ID
  • Hacking
  • Hacktivism
  • Intelligence
  • Internet of Things
  • Laws and regulations
  • Malware
  • Mobile
  • Reports
  • Security
  • Social Networks
  • Terrorism
  • ICS-SCADA
  • POLICIES
  • Contact me
  • Home
  • Cyber Crime
  • Laws and regulations
  • Security
  • Memex – The new search tool to dig also in the Deep Web

Memex – The new search tool to dig also in the Deep Web

Pierluigi Paganini February 10, 2015

DARPA has publicly presented for the first time a new set of search tools called Memex which will improve also researches into the “Deep Web”.

In 2014, the U.S. Defense Advance Research Projects Agency (DARPA) launched a the MEMEX project to design advanced search tools that could be also used to scan the deep web, which isn’t indexed by Google and other commercial search engines.

The Memex  search engine was started to allow search of not indexed content, an operation that in the majority of cases is still run manually by Intelligence Agency.

“DARPA has launched the Memex program. Memex seeks to develop the next generation of search technologies and revolutionize the discovery, organization and presentation of search results. The goal is for users to be able to extend the reach of current search capabilities and quickly and thoroughly organize subsets of information based on individual interests. Memex also aims to produce search results that are more immediately useful to specific domains and tasks, and to improve the ability of military, government and commercial enterprises to find and organize mission-critical publically available information on the Internet” states the official page of the project.

A few days ago, for the first time, the  security community has the opportunity to give a look to the MEMEX system, DARPA opened the doors of its laboratories to the journalists of the Scientific American. The Agency provided a preview of the powerful search engine and explained its use in the fight against the cybercrime.

MEMEX search engine

The Pentagon’s research agency gave Scientific American a preview of the software and 60 Minutes exclusive looks at the technology. The researchers explained that there is an impressive amount of data that is not considered useful for ordinary web users, but that represents a crucial source of information for law enforcement and intelligence agencies.

“That leaves untouched a multitude of information that may not be valuable to the average Web surfer but could provide crucial information to investigators.” states the article published by the Scientific American.

The majority of information in the Deep Web is unstructured data which are gathered from multiple sources that could not be crawled by ordinary search engines. The most popular subset of the Deep Web is the Tor network, an anonymizing network that is accessible only by using specific software.

“We’re envisioning a new paradigm for search that would tailor indexed content, search results and interface tools to individual users and specific subject areas, and not the other way around,” said Chris White, DARPA program manager. “By inventing better methods for interacting with and sharing information, we want to improve search for everybody and individualize access to information. Ease of use for non-programmers is essential.”

The DARPA involved 17 different teams of researchers, composed of representatives of the academic world and private industry, to develop most advanced technologies to include in the MEMEX program. The ambitious projects aim to revolutionize the way so search and present information from a larger pool of sources, including the content on the Deep Web.

In many circumstances, we have explained that the Deep Web is often abused by criminals that manage a multitude of illegal activities, including child pornography, human trafficking, drug deals and any other cyber criminal activities.

According to several reports, including one published by researchers at the Carnegie Mellon University, the NYDA’s Office is one of several bureaus and agencies that have already used earlier versions of the Memex system to collect information on human trafficking cases and prosecute criminals.

In a video interview released by Chris White, the inventor of the Memex search engine, the expert explained how this amazing platform works and which kind of information is able to access.

MEMEX search engine CBS video

 “The internet is much, much bigger than people think,” White said. “By some estimates Google, Microsoft Bing, and Yahoo only give us access to around 5% of the content on the Web.” said White, highlighting that the Deep Web represents a privileged environment for bad actors and their illegal affairs.

Very eloquent was the intervention of another researcher involved in the project MEMEX, the DARPA innovation head Dan Kaufman. who says, ”

“the easiest way to think about Memex is: How can I make the unseen seen?” commented Kaufman. “Most people on the internet are doing benign and good things,” Kaufman said. “But there are parasites that live on there, and we take away their ability to use the internet against us– and make the world a better place.”

At the moment, different Intelligence Agencies are testing the Memex, including two district attorneys’ offices, a law enforcement agency, and a nongovernmental organization. In the next phase the access to the platform will be extended to a greater number of beta testers to stress the capabilities of the application.

“The next set of testing begins in a few weeks and will include federal and district prosecutors, regional and national law enforcement and multiple NGOs. One of the main objectives of this round is to test new image search capabilities that can analyze photos even when portions that might aid investigators—including traffickers’ faces or a television screen in the background — are obfuscated,” Scientific American reports. “Another goal is to try out different user interfaces and to experiment with streaming architectures that assess time-sensitive data.”

White remarked the potentiality of a set of tools like MEMEX and that improvements they bring to searches conducted by law enforcement for their investigation and to prevent crimes, the experts also explained that its system will never hack any service on the Internet in order to retrieve information-

“White made several key decisions about the type of data Memex could access in an effort to steer clear of the controversy around government access to private citizen information and communications, a particularly touchy subject since Edward Snowden’s National Security Agency revelations beginning in June 2013. If something is password protected, it is not public content and Memex does not search it, according to White.” reported the Scientific American.

“We didn’t want to do hacking,” White added. “We didn’t want to cloud this work unnecessarily by dragging in the specter of snooping and surveillance.” 

As usual happen for special projects born in the Intelligence, one day we would all benefit of the services offered by MEMEX and its searches, we are drowning in a sea of information.

“Memex would ultimately apply to any public domain content; initially, DARPA intends to develop Memex to address a key Defense Department mission: fighting human trafficking. Human trafficking is a factor in many types of military, law enforcement and intelligence investigations and has a significant web presence to attract customers. The use of forums, chats, advertisements, job postings, hidden services, etc., continues to enable a growing industry of modern slavery. An index curated for the counter-trafficking domain, along with configurable interfaces for search and analysis, would enable new opportunities to uncover and defeat trafficking enterprises. Memex plans to explore three technical areas of interest: domain-specific indexing, domain-specific search, and DoD-specified applications.”

Pierluigi Paganini

(Security Affairs –  MEMEX project, DARPA)


facebook linkedin twitter

you might also like

Pierluigi Paganini July 10, 2025
UK NCA arrested four people over M&S, Co-op cyberattacks
Read more
Pierluigi Paganini July 10, 2025
Qantas data breach impacted 5.7 million individuals
Read more

leave a comment

newsletter

Subscribe to my email list and stay
up-to-date!

    recent articles

    UK NCA arrested four people over M&S, Co-op cyberattacks

    Cyber Crime / July 10, 2025

    PerfektBlue Bluetooth attack allows hacking infotainment systems of Mercedes, Volkswagen, and Skoda

    Hacking / July 10, 2025

    Qantas data breach impacted 5.7 million individuals

    Data Breach / July 10, 2025

    DoNot APT is expanding scope targeting European foreign ministries

    APT / July 10, 2025

    Nippon Steel Solutions suffered a data breach following a zero-day attack

    Data Breach / July 09, 2025

    To contact me write an email to:

    Pierluigi Paganini :
    pierluigi.paganini@securityaffairs.co

    LEARN MORE

    QUICK LINKS

    • Home
    • Cyber Crime
    • Cyber warfare
    • APT
    • Data Breach
    • Deep Web
    • Digital ID
    • Hacking
    • Hacktivism
    • Intelligence
    • Internet of Things
    • Laws and regulations
    • Malware
    • Mobile
    • Reports
    • Security
    • Social Networks
    • Terrorism
    • ICS-SCADA
    • POLICIES
    • Contact me

    Copyright@securityaffairs 2024

    We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
    Cookie SettingsAccept All
    Manage consent

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities...
    Necessary
    Always Enabled
    Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
    Non-necessary
    Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
    SAVE & ACCEPT