Working notes from the Scandinavian Institute for Computational Vandalism

Detection of Face Morphing Attacks by Deep Learning

Identification by biometric features has become more popular in the last decade. High quality video and fingerprint sensors have become less expensive and are nowadays standard components in many mobile devices. Thus, many devices can be unlocked via fingerprint or face verification. The state of the art accuracy of biometric facial recognition systems prompted even systems that need high security standards like border control at airports to rely on biometric systems. While most biometric facial recognition systems perform quite accurate under a controlled environment, they can easily be tricked by morphing attacks. The concept of a morphing attack is to create one synthetic face image that contains characteristics of two different individuals and to use this image on a document or as reference image in a database. Using this image for authentication, a biometric facial recognition system accepts both individuals. In this paper, we propose a morphing attack detection approach based on convolutional neural networks. We present an automatic morphing pipeline to generate morphing attacks, train neural networks based on this data and analyze their accuracy. The accuracy of different well-known network architectures are compared and the advantage of using pretrained networks compared to networks learned from scratch is studied.

Detection of Face Morphing Attacks by Deep Learning, Clemens Seibold, Wojciech Samek, Anna Hilsmann and Peter Eisert

Escape from the monochrome room

“Imagine a girl called Mary. She is a brilliant neuroscientist and a world expert on colour vision. But because she grew up entirely in a black and white room, she has never actually seen any colours. Many black and white books and TV programmes have taught her all there is to know about colour vision. Mary knows facts like the structure of our eyes and the exact wavelengths of light that stimulate our retinas when we look at a light blue sky.
One day, Mary escapes her monochrome room, and as she walks through the grey city streets, she sees a red apple for the first time.
What changes upon Mary’s encounter with the red apple? Has Mary learnt anything new about the colour red upon seeing the colour for the first time?”

What Did Mary Know? by Marina Gerner.

Read Gerner’s article

Dead lends a hand, murder lends glasses

In Dead Lends a Hand, the investigator Brimmer kills a woman he attempts to blackmail. The camera shows Brimmer’s face just after the murder and zooms in. After a few seconds, the image of the face becomes still as if Brimmer was paralyzed in shock. At the same time, while his face stays motionless, the narrative continues within the frame of his glasses. We see him cleaning up the room, removing his fingerprints from the objects and the victim’s body. The glasses function as a split screen showing the same action asynchronously while the face’s image stays frozen at the time of the murder, becoming a static backdrop. The glasses are the device through which he can see himself removing (every trace of) himself from the murder he just committed.

Machines to write with the hereafter

Machine à écrire avec l’au-delà, Jean Perdrizet, 1971.

Oui-Ja électrique, Jean Perdrizet, 1971.

Perdrizet drew the designs for several devices meant to communicate with the beyond. Ghosts and the dead would trigger electrical signals translated in the letters of the alphabet by Perdrizet’s inventions. As Perdrizet didn’t believe ghosts would speak a human language, he invented the T language, the sidereal Esperanto.

More about Perdrizet

When I interpret the removal of a hat

all sieves are shadows of the substances they sort

[…] we apperceive through our sieves as much as we sieve through our apperception. We appersieve, if you will. Or, if you go back to Kant ([1781] 1965), who defined the ego as the transcendental unity of apperception (whatever that means), we are our sieves.
Indeed, crucially, sieves have to take on (and not just take in ) features of the substances they sieve, if only as “inverses” of them. A hole in the ground, for example, constitutes a simple sieve: anything with a diameter less than the hole will fall through; anything with a diameter larger than the hole will stay on top. In this way, to sieve a substance, the sieve must often have an (elective) affinity with the substance to be sieved and, in particular, the qualities sieved for—in this case size. In some sense, all sieves are inverses or even shadows of the substances they sort. By necessity, they exhibit a radical kind of intimacy

Note, then, that sieves — such as spam filters—have desires built into them (inso-far as they selectively permit certain things and prohibit others); and they have beliefs built into them (insofar as they exhibit ontological assumptions). And not only do sieves have beliefs and desires built into them (and thus, in some sense, embody values that are relatively derivative of their makers and users); they may also be said to have emergent beliefs and desires (and thus embody their own relatively originary values, however unconscious they and their makers and users are of them). In particular, the values of the variables are usually steps ahead of the consciousness of the programmers (and certainly of users)—and thus constitute a kind of prosthetic unconsciousness with incredibly rich and wily temporal dynamics. Note, then, that when we make algorithms and then set those algorithms loose, there is often no way to know what’s going to happen next (Bill Maurer, personal communication).

Paul Kockelman in The anthropology of an equation, Sieves, spam filters, agentive algorithms, and ontologies of transformation

The importance of punctuation

The typesetters at Sytin’s print-works in Moscow struck on September 19. They demanded a shorter working day and a higher piecework rate per 1,000 letters set, not excluding punctuation marks.

This small event set off nothing more nor less than the all-Russian political strike – the strike which started over punctuation marks and ended by felling absolutism.

The police department plaintively reported that an association called the Union of Moscow Typographers and Lithographers, banned by the government, had taken advantage of the strike at Sytin’s. By the evening of September 24, fifty printing works were on strike. A program of claims was drawn up on the twenty-fifth at a meeting permitted by the city governor. This program was interpreted by the city governor as an “arbitrary action of the Soviet of print shop deputies,” and in the name of the personal “independence” of workers menaced by such “arbitrary” proletarian action, this police satrap tried to put down the print-workers’ strike with his clumsy fist.

But the strike which had arisen over punctuation marks had already had time to spread to other branches.

Leon Trotsky, The Soviet and the Revolution (quoted by Peter Szendy in À coups de points)

this third eye extracted from the abdomen

‘I feel a very sincere admiration for all those people who march around Spain and Italy laden with bags, cases, telemeters, extra lenses, pose—meters, thermocolorimeters (“for taking the temperature of colour”) and who, never losing the smallest clasp from their bags, or the tiniest roll of film, advance with great strides into the ‘Leican era’. Through the absurdities of people’s behaviour, one can see the attitude to culture which is particularly expressed by a certain type of tourism: ‘What I particularly fear about the “small formats” is knowing that terrible state of slavery to which they condemn an infinite number of people who would really seem to deserve a better fate. When they arrive on holiday at a beauty spot or in front of a recommended campanile, travellers immediately think of their cameras. Rather than contemplating the landscape with their frontal eyes, these people rush to have it admired by this third eye extracted from the abdomen’.
Pierre Daninos in Pierre Bourdieu’s Photography a Middle-brow Art.

Building a social detector, Mass Observation

By Professor Julian Huxley

Science in its progress is advancing nearer to the human heart of things, The first great advances in the scientific renaissance were made in the remoter and simpler fields of astronomy and physics. Then followed chemistry and a little later general biology and physiology. The great revolution in regard to individual psychology did not take place until well into the present century. Now it is the tum of the most complex of all the sciences, sociology, which is also the nearest home, since we live immersed in society as a fish in water, and our ways of thinking and
feeling are molded by the social framework.

Within the social sciences, social anthropology holds an essential place. Yet, with few exceptions, it has started to choose its material from among primitive and out-of-the-way peoples. Here again the trend must be from the remote to the near at hand. Not only scientifically but practically it is urgent to obtain detailed and unbiased information as to the mode of thinking of the larger, more powerful and economically more important groups of human beings. Most urgent of all is to obtain such knowledge about our own group, the English people.


a close listening to the technical workings of computers and their networks

In Algorhythmics: Understanding Micro-Temporality in Computational Cultures, Shintaro Miyazaki discusses the importance of rhythm to understand the performances of algorithms.

“According to Burton [chief engineer of the Manchester Small-Scale Experimental Machine-Resurrection-Project], the position of the so-called “noise probe” was variable, thus different sound sources could be heard and auscultated. These could be individual flip-flops in different registers, different data bus nodes or other streams and passages of data traffic. Not only was passive listening of the computer-operations very common, but was also an active exploration of the machine, listening to its rhythms. Burton continues,

“Very common were contact failures where a plug-in package connector to the ‘backplane’ had a poor connection. These would be detected by running the Test Program set to continue running despite failures (not always possible, of course), and then listening to the rhythmic sound while going round the machine tapping the hardware with the fingers or with a tool, waiting for the vibration to cause the fault and thus change the rhythm of the ‘tune’.”

The routine of algorhythmic listening was widespread, but quickly forgotten. One convincing reason for the lack of technical terms such as algorhythm or algorhythmic listening is the fact that the term algorithm itself was not popular until the 1960s. Additionally, in the early 1960s many chief operators and engineers of mainframe computers were made redundant, since more reliable software-based operators called operating systems were introduced. These software systems were able to listen to their own operations and algorhythms by themselves without any human intervention. Scientific standards and rules of objectivity might have played an important role as well, since listening was more an implicit form of knowledge than any broadly accepted scientific method. Diagrams, written data or photographic evidence were far more convincing in such contexts. Finally, the term rhythm itself is rarely used in the technical jargon – instead the term ‘clock’ or ‘pulse’ is preferred.”

Read the full paper

This practice of close listening has been documented at the Natuurkundig Laboratorium in Eindhoven. As explained on Ip’s Ancient Wonderworld:

“The head of the NatLab at the time, a certain Natuurkundig Laboratorium in EindhovenW. Nijenhuis, had the idea of installing a small amplifier and loudspeaker on the PASCAL, which would pick up radio frequency interference generated by the machine. Unsurprisingly, the usual course of events unfolds: While actually intended for diagnostic purposes, people quickly discovered that they could abuse the speaker to make simple music. But Mr Nijenhuis, rather than scolding his staff for the waste of precious calculation time, actually decides to record those rekengeluiden (computing noises) on a 45 rpm vinyl.”

Read more

Listen to the recordings