Technologists have used facial recognition techniques on the large archive of Parler videos filmed from the January 6 Capitol riots, Motherboard has learned. In some cases they have been able to track individual faces across different videos, pinpointing where a person was at specific points in time, potentially even if they did not use Parler themselves.
The news signals how archivists, hackers, and hobbyists continue to mine the Parler data for what they believe may be useful insights to provide law enforcement. It also highlights the fraught issue of facial recognition, which can often be inaccurate and require manual analysis to review, and more generally demonstrates the democratization of facial recognition.
Videos by VICE
“I have written a face recognition screener and processed about 900 Parler videos so far. I have pulled out about 40k faces so far,” one of the technologists claimed. Motherboard granted the person anonymity to speak more candidly about a sensitive project.
Do you know anything else about the Parler data? We’d love to hear from you. Using a non-work phone or computer, you can contact Joseph Cox securely on Signal on +44 20 8133 5190, Wickr on josephcox, OTR chat on jfcox@jabber.ccc.de, or email joseph.cox@vice.com.
Earlier this month, a hacker known as donk_enby led a project to archive the contents of right-wing social media platform Parler after the Capitol riots. Amazon subsequently cut-off service to Parler itself, making the archive and its mirrors more valuable to researchers and journalists.
The data included users’ public posts, videos, images, as well as their associated GPS data. Motherboard previously used some of that information to show Parler users posted to the network from inside military facilities. Gizmodo and technologists used a similar approach to map videos uploaded from inside the Capitol during the riots.
Now some people are turning more to the content of the videos themselves, and specifically the faces in videos taken at the Capitol.
“Step one was isolating each face indiscriminately from all of the videos,” the technologist explained. “Step two was using the face net model to group all of the faces of the same person together so that you can track them through the videos and, in effect, through location and time,” they added.
The technologist provided Motherboard with a small sample of their data and their code. The cache included a folder of faces that their tool had managed to pull from the Parler videos. A spreadsheet included a unique identifier for what the system believed was a particular person, which video they appeared in, the timestamp of the appearance, and the latitude and longitude of where the video was taken. The small spreadsheet the technologist provided showed two videos that the system said a particular person appeared in.
Some facial recognition techniques can be notoriously inaccurate, resulting in false positives or other mistakes. The data reviewed by Motherboard did appear to show faces belonging to people who were at least wearing the same clothes in multiple images, but the accuracy of the datasets writ large is unclear.
The technologist said they have shared their findings with the FBI.
Others carrying out similar projects have published their work publicly, however. One site called Faces of the Riot includes thousands of faces the creator claims came from the Parler videos. The site includes images of the faces, a code relating to the video the face was found in, and a link to the video itself.
“I hope that this will make it a lot easier for people to submit tips to the FBI because the faces are laid out in an easy to access format, with linked videos for each face to provide context,” a co-creator of Faces of the Riot told Motherboard in an online chat. They did not want to provide their real name for safety reasons.
“The purpose of linking the video to the image with just a single click is so that before a user makes a report, they have a chance to see the context surrounding the appearance of the face in the riot/on Parler,” they continued.
The site has hit some false positives, including face detections that were actually t-shirts with New York Governor Andrew Cuomo’s face emblazoned on them, according to a tweet from the project maintainer.
The posted images include faces of people who likely had nothing to do with the riots.
“TODO: Ensure all posted images are rioters (eta: no clue, might crowdsource),” the Faces of the Riot Twitter account acknowledged in a tweet. Faces of the Riot did not respond to a request for comment.
The co-creator did make a distinction between their site and other services that may allow a user to look up the identity or name of a particular person.
“I think it’s important to make the distinction that our site does NOT employ facial recognition, just facial detection. There is no attempt to tie an identity to a face for that exact reason,” they said.
Videos and other content uploaded to Parler and other social networks in and around the riots has led to accountability for those who broke into the building or committed crimes. Affidavits filed by authorities repeatedly reference social media posts, including for Riley June Williams, a woman who allegedly stole a laptop from Nancy Pelosi’s office. Journalists, researchers, and activists have also identified members of the military and law enforcement being at or near the riots.
Clearview AI, a highly controversial facial recognition tool for law enforcement that uses data scraped from social media, said that use of its app spiked after the Capitol riot, The New York Times reported.
Subscribe to our cybersecurity podcast CYBER, here.
