<section data-background="joseph-chan-264837-unsplash.jpg">
<table style="width:100%">
<tr>
<td align="right">
<h3 style="color:#ffffff">teaching machines to see like archaeologists</h3><br><br><br><br><h4>neural networks,<br> and the antiquities trades</h4></td>
</tr>
</table>
<p align="right" style="color:#ffffff"> <small> Shawn Graham<br> @electricarchaeo <br> follow along at bit.ly/sg-apr25
</p>
---
## which presumes we know how _archaeologists_ see
Note:
Sara Perry https://eprints.soton.ac.uk/344699/
---
## narrator:
## we don't
---
<section data-background="abigail-low-532799-unsplash.jpg">
Note:
perry, hamilakis on vision, seeing, reification
roopika risam on the kinds of humans our visualizations etc permit
grounded & ungrounded antiquities
try to talk about connaisseurship
pompeian wall styles
human remains
---
## What it would be _nice_ to be able to do
1. teach the machines to understand _style_
2. teach the machines to spot the differences between a forgery & something authentic
3. send the machines out there to do the spotting at a scale and speed that we could never hope to achieve with our own fleshy fingers & imperfect eyes
---
## ethical dangers
1. reification of a privileged point of view of what constitutes 'style' or 'material culture'
2. automated authentication to raise the price of antiquities and fuel the trade
3. replication of the sins of scientific racism
---
### how neural networks beget computer vision
[Alex Mordvintsev](https://twitter.com/zzznah/status/1118917356430995456)
---
## experiment #1
### eye spy the fonseca bust
---
One-shot learning
- pass an image through a neural network
- pass a different image through an identical network
- measure the difference between the results. The difference is the degree of similarity
- requires much less data
- requires careful training data
Note:
"In face recognition systems, we want to be able to recognize a person’s identity by just feeding one picture of that person’s face to the system. And, in case, it fails to recognize the picture, it means that this person’s image is not stored in the system’s database.
To solve this problem, we cannot use only a convolutional neural network for two reasons: 1) CNN doesn’t work on a small training set. 2) It is not convenient to retrain the model every time we add a picture of a new person to the system. However, we can use Siamese neural network for face recognition." https://towardsdatascience.com/one-shot-learning-face-recognition-using-siamese-neural-network-a13dcf739e
---
Training on folders of images of Roman sculpture
Pairs left to right, 1 = different, 0 = same
---
Q. Is the Fonseca Bust a Flavian Woman?
Pics of Fonseca in one folder, pics of a flavian woman in the other
---
<img src="fonseca-flav.png" height="300"/>
Looks pretty good! But wait...
---
<img src="fonseca2.png" height="400"/>
---
## experiment #2
### give me some architectural ornamentation, and make it in pompeian red
---
generative adversarial networks
<small>Dev Nag, https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f </small>
Note:
The idea is to learn what is important in an image by getting the forger to successfully create new images; the things that it makes show what the detective considered to be the signature elements.
---
etc
---
<img src="gan-pompeii.png" height="300"/>
<img src="gan-pompeii2.png" height="300"/>
This _looks_ good, but the same visions persist, and it's actually just learned to reproduce the training set. It is not _composing_ new images. Training set was too small.
---
Much bigger sample from our bone trade research:
---
## experiment #3
### is this a skull I see before me?
---
<small> 10k images tagged with 'skulls for sale' and similar, collected Feb 2019, visualized with Pix-Plot, Yale DH lab, [code repo](https://github.com/YaleDHLab/pix-plot) </small>
---
---
---
<img src="bone-lust-eigenvector-illustration.png" height = "400" />
Note:
Time stamps. Is there directionality in the influence?
left vertex of triangle: Fri Feb 26 18:57:31
next in tail: Tue Mar 11 23:19:25
next in tail: Sat May 10 20:09:42
last in tail: Sat Apr 12 21:40:37
---
I can do the same again with human remains. But I won't be displaying those pictures.
---
A toy skull:
<img src="/papers/fields-02-19/fig1-huffer-wood-graham-conch.png" height="200"/>
<img src="/papers/fields-02-19/fig2-huffer-wood-graham-mask.png" height="200"/>
---
there. are. no. giraffes.
---
# So What?
Note:
teaching the machines to see works in a way that suggests greater authority than is warranted, which is dangerous.
---
<small>Bailey, J. 2017 'Machine learning for art valuation. An interview with Ahmed Hosny', Artnome.com, https://www.artnome.com/news/2017/12/2/machine-learning-for-art-valuation </small>
Note:
Already, people are using neural networks to identify 'high value' artworks so as to set reserve prices
---
<small>Hirschman, Dan. artificial intelligence discovers gayface. sigh. https://scatter.wordpress.com/2017/09/10/guest-post-artificial-intelligence-discovers-gayface-sigh/</small>
Note:
Already, people are using neural networks to identify sexual orientation, or presence/absence in criminal databases
---
<section data-background="josh-marshall-133093-unsplash.jpg">
Note:
Already, the use of these techniques reifies assumptions about culture, origin, and value that dehumanize and demean the living _and the dead_.
---
<section data-background="kyle-glenn-336141-unsplash.jpg">
Note:
The use of this technology in the context of antiquities trading, human remains trading, forgeries is going to throw up many _edge cases_ that will require human intervention to sort out. This may improve the algorithm: but will it make it ethical?
An ethical use case: taxonomic processes. Instead of authenticating, let the machines look for hints of context. Let the machines look for low-stakes features.
---
<section data-background="callum-shaw-557874-unsplash.jpg">
Note:
Leave the complex reasoning to the humans.
Neural networks are too easily spoofed while also carrying too much rhetorical power.
---
<section data-background="callum-shaw-557874-unsplash.jpg">
<h3 align="left">thank you</h3>
<p align="left">shawn.graham@carleton.ca</p>
<p align="left"><small>This research has been generously supported by the<br> Social Sciences and Humanities Research Council of Canada</small>
<h3 align="left">[bonetrade.github.io](http://bonetrade.github.io) </h3>
---
<small>full slide photo credits, unsplash.com:
joseph chan
josh marshall
kyle glenn
callum shaw
abigail low