The hidden pattern within your digital camera

Humans have always looked for deeper meaning in imagery; from Disney films, to religious iconography, and the Mona Lisa's crooked smile. But there is, in fact, a pattern hidden in every digital photograph and video frame -- image fingerprints that are being put to novel uses in social media, mobile authentication and criminal forensics.

Even with the most advanced manufacturing methods, every camera sensor is not created equal.

For example, a 10 megapixel camera sensor has an array of 10 million pixels -- tiny cavities which trap light, turning it into a digital signal -- and each of these can vary in how it measures light.

In an ideal world, photographs of a pure blue sky taken by two identical cameras would be indistinguishable. In reality, this is not the case. The pixel imperfections of a camera's sensor leave a unique fingerprint on the images it creates, known as sensor pattern noise.

The research enabling the extraction of these weak, but universal traces already emerged in 2006, but several problems remained to be solved. For one, the extracted fingerprints dwarf their original images, hindering large-scale sensor pattern noise applications due to the brute computational force and extensive storage space required.

Searches

But a team of Italian researchers working on a project named CRISP, developing technology for compressing signals and images in big data, stumbled onto a potential solution.

Professor Enrico Magli, of Politecnico di Torino, is using the compression technology of this project, which was funded with a grant from the EU's European Research Council (ERC), to streamline the identification of the cameras behind online images.

He launched a separate ERC-funded project, known as ToothPic, in late 2015, to look specifically at the camera identification issue.

'What we have in mind with ToothPic is the ability to be able, with a suitably sized data centre, to process millions of pictures per second,' said Prof. Magli. 'What we do is really to make the fingerprint easily searchable and optimised to make it as fast as possible.'

Primarily, ToothPic is zooming in on the potential to verify image copyrights.

Prof. Magli says copyright infringement is now something of a 'national sport' on photo-based social media websites like Flickr. With more than 350 million pictures also uploaded to Facebook daily, he contends that there needs to be a way to deal with illegal or unethical photo use in this sphere.

To this end, his team will launch a demonstration search engine this summer, and has a network of about 100 university computers hard at work, downloading 50 million pictures from Flickr.

The demo -- a simple internet app, where people can upload a picture or fingerprint and check if it matches photos in ToothPic's database -- is hoped to attract the attention of social media heavyweights like Google or Facebook, which could theoretically offer an internet-wide fingerprint search service thanks to their extensive image databases.

While Prof. Magli notes there are privacy concerns around using this kind of information, he believes such photo fingerprint applications are nothing to be concerned about.

ToothPic's technology has already sparked a media forensics collaboration with the Turin Police, while photo marketplaces and photographers seeking their stolen cameras are also interested, he said.

The team will simultaneously launch the software through a startup company, and hopes to add a parallel product by autumn, which uses the sensor pattern noise from smartphone cameras to offer a simpler alternative to two-step authentication processes like Wi-Fi logins.

Forensics

In the UK, a research and industry consortium named DIVeFor has already proved the worth of sensor pattern noise technology in criminal forensics, in 2014 helping Sussex Police to secure the first ever conviction using image fingerprints.

Professor Chang-Tsun Li, of the University of Warwick, led the EU-funded project, a major focus of which was developing multimedia sensor pattern noise extraction technologies. When police arrested a man on suspicion of possessing child pornography and discovered the suspect's mobile phone held two videos involving children, they turned to Prof. Li's technology.

'Initially, the suspect denied his responsibility, and claimed that the two videos were sent to him by someone else,' he explained.

Police identified the fingerprint of the smartphone camera, and by comparing this to the fingerprints of the pornographic video stills, found a match strong enough that the man pleaded guilty and was sentenced to nine years in jail.

The professor is now using sensor pattern noise technology to help Interpol group the 5 million images in its international child exploitation database by source camera. He says this is a much more challenging prospect, because without possessing the original cameras, these images can only be grouped by the similarities between their very weak fingerprints.

To perform the complex task of tying these similarities together, he has had to develop machine learning software -- an artificial intelligence that can recognise patterns and use this information in a way normal modelling cannot.

With the potential of this technology in mind, Prof. Li this year founded the IDENTITY research consortium, with EU Marie Skłodowska-Curie grant backing. This international effort combines industry and academic partners from 12 European, Asian and American countries, as well as senior forensic police as advisors.

'On this project we are not only trying to identify devices, but we also want to use biometric techniques to identify people through physiological and behavioural traits,' he said.

'No matter if you are going to identify devices or identify people, the underlying, enabling technologies are similar -- it's computer vision, machine learning and pattern recognition. Therefore, whatever technique we develop for device identification can potentially be fine-tuned or modified to serve the purpose of people identification, or vice versa.'