# Exploring Art, Theater and Sentence Entropy

# with Machine Learning

---

# Hello!

## I’m Miles Thompson

Mostly I’m a data scientist and developer but sometimes I like to pretend I’m an artist ;-)  
You can find me at [miles@goodbright.nz](mailto:miles@goodbright.nz)

---

## The great outdoors

---

## Some Motivation

---

## Table of Contents

# ◉ Context ◉ Data collection  
# ◉ Entropy ◉ Other experiments

---

Computers  
Machine Learning

---

## About the director,

## Annie Dorsen

**Broadway** – “Passing Strange”

## Algorithmic Theater

- “Hello Hi There”  
- “Yesterday Tomorrow”  
- “The Great Outdoors”

---

“The audience will gather around the flickering projector, with all kinds of text and stories summoned down to us from the “cloud.” .. We gather together to hear the stories not so much from our ancestors as from each other, right now, two seconds ago.”

**“**

---

## Development process

---

Gear 1  
Gear 2

Some of the people involved
◉  
◉ Director  
◉ Music programming  
◉ Video programming  
◉ Text programming  
◉ Dramaturgy  
◉ Our amazing actor/s  
◉

---

The theater process  
◉ Theater - turns out “it’s a whole thing” ;-)  
◉ Need very rapid iterations  
( < 1 hour)  
◉ Solution: Split code between preparation and script.  
Adjust “script” code during rapid iteration, do numeric processing offline.

---

## Data Collection.

---

# Pushshift.io

---

# Spark / Databricks

Power but $$

---

# Motorway

## more home-build but free

---

# DEMO

---

A quick dive into some of the text

---

**Entropy**

---

# The Arc of the piece

---

Lowest entropy

---

Lowish entropy

---

Middle entropy / More personal, long stories

---

Higher entropy

---

Even higher entropy

---

Very highest entropy

---

**Shannon’s definition of entropy**

# ◉.. minimum number of bits needed to encode  
a string of symbols, based on the frequency of the symbols ..  
# ◉ “ The form of H will be recognized as that of  
_entropy as defined in certain formulations_ _of statistical mechanics where p_ _i_ _is the_ _probability of a system being in cell i of its_ _phase space.”_ _-Bell System Technical Journal, 1948_ _“A Mathematical Theory of Communication“_ _Shannon, C.E._

---

**Shannon’s definition of entropy**

# ◉.. minimum number of bits needed to encode  
a string of symbols, based on the frequency of the symbols ..  
# ◉ “ The form of H will be recognized as that of  
_entropy as defined in certain formulations_ _of statistical mechanics where p_ _i_ _is the_ _probability of a system being in cell i of its_ _phase space.”_ _-_ _Bell System Technical Journal, 1948_ _“A Mathematical Theory of Communication“ Shannon,_ 
_C.E._

---

Quantifying a word's information content

entropy: how uncertain we are about what is being communicated at a certain point (it decreases with each upcoming word)

POS  
INTERP  
SIBLE RETATIONS  
maximal uncertainty: all interpretations are equally probable  
we are certain about one particular interpretation  （ $ P ( x )=1 $ ）  
after the first w

---

# Shannon Entropy, very bad entropy and other measures

## ◉ Just counting how common a sentence is  
◉ Shannon entropy

## ◉ Compression entropy (online compression)  
◉ “Very bad entropy” (PySmaz)

---

# Semantic‘Entropy – RNN Entropy

---

# Semantic‘Entropy – W2V Entropy’

---

# Sentence online with highest W2V Entropy?

## James Joyce – Ulysses-for the win

---

## Other experiments

---

# Content Shaping

---

# Semantic Resonance

---

# Rhyming

---

## On working with artists

---

## Working with artists

# ◉ Don’t Dunning Kruger it
- realize how much you don’t know

# ◉ Don’t show algorithms without being
cognizant of other aesthetics – wait a bit more till ready?

# ◉ “Open source” ?
- can make some artists twitch

# ◉ Respect
- is spelled R E S P E C T

---

_.. we live in a world in which the question of agency is highly disputed, in which our access to choice is circumscribed by the framing of political discourse, by the simultaneous cornucopia and contraction of “options” in our consumer paradise, and, indeed, by algorithms, which filter, consolidate and display certain possibilities while rendering all others invisible._

- Annie Dorsen, 2012  
**“**

---

Thanks!

Any questions?  
You can find me at  
◉  
@utunga  
[miles@goodbright.nz](mailto:miles@goodbright.nz)

[miles@goodbright.nz](mailto:miles@goodbright.nz)  
[www.goodbright.nz](/content/site-root.html)