Sucking Less - interp

Sucking Less - interp Notes on sucking less. Zola 2026-05-24T00:00:00+00:00 https://ryqford.xyz/tags/interp/atom.xml Small Models Have Emotions, Too! 2026-05-24T00:00:00+00:00 2026-05-24T00:00:00+00:00 Ryan Ford https://ryqford.xyz/blog/small-models-have-emotions-too/ <blockquote> <p>This is an ongoing experiment and if you are reading this post, please be aware it is still in the draft stage.</p> </blockquote> <p>Anthropic <a rel="external" href="https://transformer-circuits.pub/2026/emotions/index.html">recently published</a> some work on emotions and their functions in LLMs. They find that Sonnet 4.5 develops structured internal representations of human emotions. Does this occur in smaller open models too, though? In this post we search for them in Qwen-2.5-7B, and further investigate how these emotion vectors change across languages. Specifically, we begin an investigation of how the internal geometry of a language’s emotion space compares to other languages.</p> <blockquote> <p>This post was inspired by vogel’s <a rel="external" href="https://vgel.me/posts/qwen-introspection/">Small Models Can Introspect, Too</a> and compute was partially funded by BlueDot Impact.</p> </blockquote> <h2 id="introduction">Introduction</h2> <p>Recent interpretability research (e.g., Anthropic’s investigations into emotion circuits) has demonstrated that LLMs develop structured internal representations of human emotions. In this independent research project we attempt to reproduce these findings in smaller, accessible models (e.g., Qwen-2.5 7B).</p> <p>Building on this foundation, we aim to investigate a novel objective: <strong>How do these emotion vectors relate across different languages?</strong> Specifically, we seek to determine whether the internal representation space for a set of emotions in one language is isomorphic to that of another. An additional question we propose is if LLMs learn a universal, language-agnostic “concept space” for emotions, or if these representations shift based on the cultural and linguistic nuances of the training data.</p> <p>Anthropic was very thorough in making their work easy to reproduce, giving detailed information and publishing the prompts they used to create their datasets publicly.</p> <p>We will first investigate a single language pair: Japanese and English. These languages provide a good ground for comparison, as they are syntactically very different, share no input tokens, and the author is able to evaluate the results of both languages. Qwen was chosen over Llama for its more robust Japanese performance.</p> <p>Our plan is as follows.</p> <ol> <li>Create dataset of contrastive pairs <ol> <li>Write short stories on diverse topics in which a character experiences a specified emotion</li> <li>Write neutral dialogues about the same prompted topics, but void of emotional expressivity.</li> </ol> </li> <li>Extract activations <ol> <li>Extract residual stream activations at each layer, averaged across all token positions, beginning from the 50th token.</li> <li>Average these activations across stories with the same emotion, and subtract mean from all emotions</li> </ol> </li> <li>Eliminate confounds <ol> <li>Obtain activations from neutral stories, compute top principal components (enough to account for 50% of the variance)</li> <li>Project out these components from the emotion vectors</li> </ol> </li> </ol> <h2 id="dataset-creation">Dataset Creation</h2> <p>For representation engineering, we must first create a dataset of contrastive pairs. In our case, the contrastive pairs will first not be against a neutral baseline, but rather contrastive between a wide variety of different emotions. Following Anthropic’s example, we also generate emotion-neutral data points, but have not yet been able to achieve comparible results in their usage (where the authors reduced the variance of up to 50%).</p> <p>We reuse Anthropic’s 100 topics and generate 12 stories per topic per emotion per language. Our prompts can be found here. To reduce costs, we only generate stories for 30 emotions with Sonnet 4.6. Opus 4.7 helped the author to find a good representative subset of emotions to use, focusing on having a balanced set across both valence and arousal.</p> <p>It’s important to note that the Japanese and English stories are not translations of each other– the author initially (and naively) wanted to test if cultural shift in emotions was measurable, which requires independent and native-like prose.</p> <h2 id="extraction-and-sanity-checks">Extraction and Sanity Checks</h2> <h3 id="collecting-activations">Collecting Activations</h3> <p>To collect activations for our stories, we do a forward pass on each story, extracting the residual stream at each layer, taking the activations at each token after the 50th token and averaging them together.</p> <p>For each story, we collect an activation by doing a forward pass and, for each layer, taking the activations of all tokens after the 50th token (where the emotional context is likely already established) and taking the mean. We do this for each story for an emotion, then average across all story points to get the raw vector for that emotion.</p> <p>Below is a slightly simplified version of the code used, the complete version can be found on GitHub.</p> <pre class="giallo z-code"><code data-lang="python"><span class="giallo-l"><span class="z-storage">def</span><span class="z-entity z-name z-function"> get_mean_activations_for_texts</span><span>(</span><span>model</span><span>,</span><span> texts</span><span>,</span><span> batch_size</span><span class="z-keyword">=</span><span class="z-constant z-numeric">8</span><span>)</span><span>:</span></span> <span class="giallo-l"><span class="z-string"> """</span></span> <span class="giallo-l"><span class="z-string"> Runs texts through the model, extracts residual stream activations at each layer,</span></span> <span class="giallo-l"><span class="z-string"> averages across positions >= 50, and returns the mean across texts.</span></span> <span class="giallo-l"><span class="z-string"> """</span></span> <span class="giallo-l"><span> n_layers</span><span class="z-keyword"> =</span><span> model</span><span>.</span><span>cfg</span><span>.</span><span>n_layers</span></span> <span class="giallo-l"><span> d_model</span><span class="z-keyword"> =</span><span> model</span><span>.</span><span>cfg</span><span>.</span><span>d_model</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span> sum_activations</span><span class="z-keyword"> =</span><span> torch</span><span>.</span><span>zeros</span><span>(</span><span>(</span><span>n_layers</span><span>,</span><span> d_model</span><span>)</span><span>,</span><span> device</span><span class="z-keyword">=</span><span class="z-string">'</span><span class="z-string">cpu</span><span class="z-string">'</span><span>)</span></span> <span class="giallo-l"><span> names_filter</span><span class="z-keyword"> =</span><span class="z-storage"> lambda</span><span> name</span><span>:</span><span> name</span><span>.</span><span>endswith</span><span>(</span><span class="z-string">"</span><span class="z-string">resid_post</span><span class="z-string">"</span><span>)</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> for</span><span> i</span><span class="z-keyword"> in</span><span> tqdm</span><span>(</span><span class="z-support z-function">range</span><span>(</span><span class="z-constant z-numeric">0</span><span>,</span><span class="z-support z-function"> len</span><span>(</span><span>texts</span><span>)</span><span>,</span><span> batch_size</span><span>)</span><span>,</span><span> leave</span><span class="z-keyword">=</span><span class="z-constant z-language">False</span><span>)</span><span>:</span></span> <span class="giallo-l"><span> batch_texts</span><span class="z-keyword"> =</span><span> texts</span><span>[</span><span>i</span><span>:</span><span>i</span><span class="z-keyword">+</span><span>batch_size</span><span>]</span></span> <span class="giallo-l"><span> tokens</span><span class="z-keyword"> =</span><span> model</span><span>.</span><span>to_tokens</span><span>(</span><span>batch_texts</span><span>)</span></span> <span class="giallo-l"><span> seq_len</span><span class="z-keyword"> =</span><span> tokens</span><span>.</span><span>shape</span><span>[</span><span class="z-constant z-numeric">1</span><span>]</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> with</span><span> torch</span><span>.</span><span>no_grad</span><span>(</span><span>)</span><span>:</span></span> <span class="giallo-l"><span> _</span><span>,</span><span> cache</span><span class="z-keyword"> =</span><span> model</span><span>.</span><span>run_with_cache</span><span>(</span><span>tokens</span><span>,</span><span> names_filter</span><span class="z-keyword">=</span><span>names_filter</span><span>,</span><span> return_type</span><span class="z-keyword">=</span><span class="z-constant z-language">None</span><span>)</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span> batch_size_actual</span><span class="z-keyword"> =</span><span> tokens</span><span>.</span><span>shape</span><span>[</span><span class="z-constant z-numeric">0</span><span>]</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> for</span><span> b</span><span class="z-keyword"> in</span><span class="z-support z-function"> range</span><span>(</span><span>batch_size_actual</span><span>)</span><span>:</span></span> <span class="giallo-l"><span class="z-comment"> #</span><span class="z-comment"> ignoring pad tokens</span></span> <span class="giallo-l"><span> valid_len</span><span class="z-keyword"> =</span><span> (</span><span>tokens</span><span>[</span><span>b</span><span>]</span><span class="z-keyword"> !=</span><span> model</span><span>.</span><span>tokenizer</span><span>.</span><span>pad_token_id</span><span>)</span><span>.</span><span>sum</span><span>(</span><span>)</span><span>.</span><span>item</span><span>(</span><span>)</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> for</span><span> l</span><span class="z-keyword"> in</span><span class="z-support z-function"> range</span><span>(</span><span>n_layers</span><span>)</span><span>:</span></span> <span class="giallo-l"><span> layer_name</span><span class="z-keyword"> =</span><span class="z-storage"> f</span><span class="z-string">"</span><span class="z-string">blocks.</span><span class="z-constant z-character">{</span><span>l</span><span class="z-constant z-character">}</span><span class="z-string">.hook_resid_post</span><span class="z-string">"</span></span> <span class="giallo-l"><span class="z-comment"> #</span><span class="z-comment"> Mean from 50th token to valid_len</span></span> <span class="giallo-l"><span> act</span><span class="z-keyword"> =</span><span> cache</span><span>[</span><span>layer_name</span><span>]</span><span>[</span><span>b</span><span>,</span><span class="z-constant z-numeric"> 50</span><span>:</span><span>valid_len</span><span>,</span><span> :</span><span>]</span><span>.</span><span>mean</span><span>(</span><span>dim</span><span class="z-keyword">=</span><span class="z-constant z-numeric">0</span><span>)</span><span>.</span><span>cpu</span><span>(</span><span>)</span></span> <span class="giallo-l"><span> sum_activations</span><span>[</span><span>l</span><span>]</span><span class="z-keyword"> +=</span><span> act</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span> total_valid_texts</span><span class="z-keyword"> +=</span><span class="z-constant z-numeric"> 1</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> del</span><span> cache</span></span> <span class="giallo-l"><span> torch</span><span>.</span><span>cuda</span><span>.</span><span>empty_cache</span><span>(</span><span>)</span></span> <span class="giallo-l"><span> </span></span> <span class="giallo-l"><span class="z-keyword"> return</span><span> sum_activations</span><span class="z-keyword"> /</span><span> total_valid_texts</span></span></code></pre> <p>To collect our raw emotion vectors, we simply loop through the emotions:</p> <pre class="giallo z-code"><code data-lang="python"><span class="giallo-l"><span>emotion_vectors</span><span class="z-keyword"> =</span><span> {</span><span class="z-string">'</span><span class="z-string">en</span><span class="z-string">'</span><span>:</span><span> {</span><span>}</span><span>,</span><span class="z-string"> '</span><span class="z-string">ja</span><span class="z-string">'</span><span>:</span><span> {</span><span>}</span><span>}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span class="z-keyword">for</span><span> lang</span><span class="z-keyword"> in</span><span> [</span><span class="z-string">'</span><span class="z-string">en</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">ja</span><span class="z-string">'</span><span>]</span><span>:</span></span> <span class="giallo-l"><span class="z-support z-function"> print</span><span>(</span><span class="z-storage">f</span><span class="z-string">"</span><span class="z-constant z-character">\n</span><span class="z-string">Extracting activations for language: </span><span class="z-constant z-character">{</span><span>lang</span><span class="z-constant z-character">}</span><span class="z-string">"</span><span>)</span></span> <span class="giallo-l"><span class="z-keyword"> for</span><span> emotion</span><span class="z-keyword"> in</span><span> emotions_list</span><span>:</span></span> <span class="giallo-l"><span> texts</span><span class="z-keyword"> =</span><span> texts_by_emotion</span><span>[</span><span>lang</span><span>]</span><span>[</span><span>emotion</span><span>]</span></span> <span class="giallo-l"><span> avg_act</span><span class="z-keyword"> =</span><span> get_mean_activations_for_texts</span><span>(</span><span>model</span><span>,</span><span> texts</span><span>,</span><span> batch_size</span><span class="z-keyword">=</span><span class="z-constant z-numeric">4</span><span>)</span></span> <span class="giallo-l"><span> emotion_vectors</span><span>[</span><span>lang</span><span>]</span><span>[</span><span>emotion</span><span>]</span><span class="z-keyword"> =</span><span> avg_act</span></span></code></pre><h3 id="centering">Centering</h3> <p>The collected raw emotion vectors have a lot of shared latents that we attempt to reduce (for geometrical interpretations) by centering them, meaning subtracting the mean emotion vector from each raw vector.</p> <pre class="giallo z-code"><code data-lang="python"><span class="giallo-l"><span>centered_emotion_vectors</span><span class="z-keyword"> =</span><span> {</span><span class="z-string">'</span><span class="z-string">en</span><span class="z-string">'</span><span>:</span><span> {</span><span>}</span><span>,</span><span class="z-string"> '</span><span class="z-string">ja</span><span class="z-string">'</span><span>:</span><span> {</span><span>}</span><span>}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span class="z-keyword">for</span><span> lang</span><span class="z-keyword"> in</span><span> [</span><span class="z-string">'</span><span class="z-string">en</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">ja</span><span class="z-string">'</span><span>]</span><span>:</span></span> <span class="giallo-l"><span class="z-comment"> #</span><span class="z-comment"> Shape: (num_emotions, n_layers, d_model)</span></span> <span class="giallo-l"><span> all_emotions_stack</span><span class="z-keyword"> =</span><span> torch</span><span>.</span><span>stack</span><span>(</span><span class="z-support z-type">list</span><span>(</span><span>emotion_vectors</span><span>[</span><span>lang</span><span>]</span><span>.</span><span>values</span><span>(</span><span>)</span><span>)</span><span>)</span></span> <span class="giallo-l"><span> mean_emotion_vector</span><span class="z-keyword"> =</span><span> all_emotions_stack</span><span>.</span><span>mean</span><span>(</span><span>dim</span><span class="z-keyword">=</span><span class="z-constant z-numeric">0</span><span>)</span></span> <span class="giallo-l"><span class="z-keyword"> for</span><span> emotion</span><span>,</span><span> vec</span><span class="z-keyword"> in</span><span> emotion_vectors</span><span>[</span><span>lang</span><span>]</span><span>.</span><span>items</span><span>(</span><span>)</span><span>:</span></span> <span class="giallo-l"><span> centered_emotion_vectors</span><span>[</span><span>lang</span><span>]</span><span>[</span><span>emotion</span><span>]</span><span class="z-keyword"> =</span><span> vec</span><span class="z-keyword"> -</span><span> mean_emotion_vector</span></span></code></pre><h3 id="projection">Projection</h3> <p>Because all emotions share a set of topics, and stories will inevitably contain a lot of shared latents, we follow Anthropic in generating a set of neutral dialogues. The top PCA components are then taken from the emotion vectors and projected out of the centered emotion vectors.</p> <p>Like Anthropic, we find that it slightly denoises some of the results, but qualitative results still hold using the centered emotion vectors. Unlike Anthropic, who used as many PCA components as needed to explain 50% of the variance, we found 96.42% and 94.82% variance explanation with 2 components for EN and JA, respectively.</p> <h3 id="logits">Logits</h3> <p>For an initial sanity check we take a look at the top/bottom logits for each emotion in each language. Here we find a lot of junk in English, especially cross-contamination with Chinese and German, but qualitatively can see the right direction for most of the emotions.</p> <p>The Japanese quality is substantially worse. There is even MORE cross-lingual contamination than in English, with Chinese SEO/spam phrases that have nothing to do with the emotion.</p> <details> <summary>English</summary> <table><thead><tr><th>Emotion</th><th>Top Tokens</th><th>Bottom Tokens</th><th>Notes</th></tr></thead><tbody> <tr><td>AFRAID</td><td><code>terror</code>, <code>panic</code>, <code>恐惧</code>, <code>paranoia</code>, <code>Cbd</code></td><td><code>CircularProgress</code>, <code>proudly</code>, <code>inspirational</code>, <code>admired</code>, <code>Enjoy</code></td><td></td></tr> <tr><td>ANGRY</td><td><code>愤怒</code>, <code>怒</code>, <code>骂</code>, <code>愤</code>, <code>bindActionCreators</code></td><td><code>adventures</code>, <code>curious</code>, <code>mysteries</code>, <code>intriguing</code>, <code>fond</code></td><td>Top is mostly Chinese; <code>bindActionCreators</code> is bizarre</td></tr> <tr><td>CALM</td><td><code>leaf</code>, <code>nearby</code>, <code>intriguing</code>, <code>leisure</code>, <code>tid</code></td><td><code>Fuck</code>, <code>fuck</code>, <code>fucking</code>, <code>freaking</code>, <code>gangbang</code></td><td>Bottom is a clean inversion — profanity = not calm</td></tr> <tr><td>DISGUSTED</td><td><code>clen</code>, <code>vom</code>, <code>disgust</code>, <code>nause</code>, <code>disgusting</code></td><td><code>abhängig</code>, <code>Geschäfts</code>, <code>Kunden</code>, <code>CircularProgress</code>, <code>Mitarbeiter</code></td><td>Top tokens are truncated stems? (<code>vom</code>it, <code>nause</code>a, <code>clen</code>ch)</td></tr> <tr><td>EXCITED</td><td><code>excited</code>, <code>兴奋</code>, <code>excitement</code>, <code>!\n</code>, <code>…\n</code></td><td><code>ujet</code>, <code>resden</code>, <code>creampie</code>, <code>urette</code>, <code>lke</code></td><td></td></tr> <tr><td>GUILTY</td><td><code>Mitarbeiter</code>, <code>Geschäfts</code>, <code>Gespr</code>, <code>AuthenticationService</code>, <code>Antworten</code></td><td><code>panorama</code>, <code>bâtiment</code>, <code>嗞</code>, <code>Mediterr</code>, <code>grille</code></td><td>German business cluster</td></tr> <tr><td>ASHAMED</td><td><code>AuthenticationService</code>, <code>Gespr</code>, <code>Mitarbeiter</code>, <code>esteem</code>, <code>Förder</code></td><td><code>sun</code>, <code>humming</code>, <code>sunlight</code>, <code>rhythm</code>, <code>leaf</code></td><td>Same German cluster; bottom is nature/serenity</td></tr> <tr><td>LOVING</td><td><code>loving</code>, <code>kisses</code>, <code>love</code>, <code>sweet</code>, <code>her</code></td><td><code>gangbang</code>, <code>shemale</code>, <code>Shemale</code>, <code>creampie</code>, <code>ignet</code></td><td>qwen transphobia :(</td></tr> <tr><td>SAD</td><td><code>loneliness</code>, <code>lonely</code>, <code>empty</code>, <code>sometimes</code>, <code>sleeping</code></td><td><code>Gratuit</code>, <code>CircularProgress</code>, <code>Rencontre</code>, <code>Mitgli</code>, <code>Prostit</code></td><td></td></tr> <tr><td>SURPRISED</td><td><code>ogui</code>, <code>utow</code>, <code>mó</code>, <code>@student</code>, <code>ucz</code></td><td><code>sometimes</code>, <code>often</code>, <code>whenever</code>, <code>later</code>, <code>weekends</code></td><td>bottom being temporal is interesting, makes intuitive sense</td></tr> <tr><td>JOYFUL</td><td><code>gigg</code>, <code>joy</code>, <code>laughing</code>, <code>ecstatic</code>, <code>joyful</code></td><td><code>creampie</code>, <code>shemale</code>, <code>ActionTypes</code>, <code>AuthenticationService</code>, <code>gangbang</code></td><td></td></tr> <tr><td>THRILLED</td><td><code>疯狂</code>, <code>兴奋</code>, <code>excited</code>, <code>exhilar</code>, <code>excitement</code></td><td><code>特价</code>, <code>悻</code>, <code>锃</code>, <code>BrowserAnimationsModule</code>, <code>性价</code></td><td></td></tr> <tr><td>CONTENT</td><td><code>delight</code>, <code>sun</code>, <code>enjoying</code>, <code>sunny</code>, <code>warm</code></td><td><code>AuthenticationService</code>, <code>ogui</code>, <code>füh</code>, <code>ActionTypes</code>, <code>(![</code></td><td>Top = warmth and sunshine</td></tr> <tr><td>GRATEFUL</td><td><code>years</code>, <code>loving</code>, <code>loved</code>, <code>hugs</code>, <code>joy</code></td><td><code>勠</code>, <code>企图</code>, <code>pisa</code>, <code>ủa</code>, <code>avanaugh</code></td><td></td></tr> <tr><td>PROUD</td><td><code>proudly</code>, <code>proud</code>, <code>自豪</code>, <code>pride</code>, <code>accol</code></td><td><code>Fuck</code>, <code>勠</code>, <code>ederland</code>, <code>bish</code>, <code>eneg</code></td><td></td></tr> <tr><td>HOPEFUL</td><td><code>sketch</code>, <code>sketches</code>, <code>promising</code>, <code>hopeful</code>, <code>plans</code></td><td><code>otron</code>, <code>Fuck</code>, <code>嚆</code>, <code>buồn</code>, <code>:normal</code></td><td><code>sketch</code>/<code>plans</code> = forward-looking intent</td></tr> <tr><td>NOSTALGIC</td><td><code>summers</code>, <code>nostalgia</code>, <code>nostalg</code>, <code>faded</code>, <code>当年</code></td><td><code>ASAP</code>, <code>verbally</code>, <code>calmly</code>, <code>LinkedIn</code>, <code>proactive</code></td><td>Bottom = urgency/professionalism, antithetical to reminiscing</td></tr> <tr><td>BORED</td><td><code>勠</code>, <code>incy</code>, <code>慵</code>, <code>Netflix</code>, <code>Gratuit</code></td><td><code>her</code>, <code>tears</code>, <code>herself</code>, <code>those</code>, <code>she</code></td><td><code>Netflix</code> as boredom marker; bottom = emotional narrative</td></tr> <tr><td>FRUSTRATED</td><td><code>bindActionCreators</code>, <code>ActionTypes</code>, <code>UIControl</code>, <code>gangbang</code>, <code>createAction</code></td><td><code>glimps</code>, <code>warmth</code>, <code>memories</code>, <code>fond</code>, <code>surprised</code></td><td>Top is all React/Redux boilerplate — debugging frustration?</td></tr> <tr><td>ANXIOUS</td><td><code>phanumeric</code>, <code>FixedUpdate</code>, <code>gangbang</code>, <code>焦虑</code>, <code>ogui</code></td><td><code>proudly</code>, <code>fond</code>, <code>delighted</code>, <code>joy</code>, <code>celebration</code></td><td>Unity game engine tokens (<code>FixedUpdate</code>)</td></tr> <tr><td>JEALOUS</td><td><code>Bravo</code>, <code>Fotos</code>, <code>Gorgeous</code>, <code>coveted</code>, <code>Giov</code></td><td><code>yleft</code>, <code>difficoltà</code>, <code>lse</code>, <code>镗</code>, <code>ntl</code></td><td><code>coveted</code> makes sense; rest is noisy</td></tr> <tr><td>LONELY</td><td><code>loneliness</code>, <code>evenings</code>, <code>weekdays</code>, <code>weekends</code>, <code>weekday</code></td><td><code>…</code>, <code>……</code>, <code>…\n</code>, <code>…\n\n</code>, <code>……\n\n</code></td><td>Time-of-day tokens = lonely routines</td></tr> <tr><td>EMBARASSED</td><td><code>incerely</code>, <code>lijah</code>, <code>尴尬</code>, <code>WHATSOEVER</code>, <code>leanor</code></td><td><code>mornings</code>, <code>nights</code>, <code>Sundays</code>, <code>weekdays</code>, <code>day</code></td><td></td></tr> <tr><td>CONTEMPTUOUS</td><td><code>Gratuit</code>, <code>Veranst</code>, <code>/mock</code>, <code>Prostit</code>, <code>/goto</code></td><td><code>:".$</code>, <code>midnight</code>, <code>till</code>, <code>:block</code>, <code>until</code></td><td></td></tr> <tr><td>RESENTFUL</td><td><code>Geschäfts</code>, <code>Veranst</code>, <code>Mitarbeiter</code>, <code>ActionTypes</code>, <code>AngularFire</code></td><td><code>startled</code>, <code>:block</code>, <code>nearby</code>, <code>trembling</code>, <code>unfamiliar</code></td><td>German business + Angular — same artifact cluster</td></tr> <tr><td>MELANCHOLY</td><td><code>sometimes</code>, <code>sometimes</code>, <code>lke</code>, <code>loneliness</code>, <code>nesota</code></td><td><code>"</code>, <code>"</code>, <code>…</code>, <code>…\n</code>, <code>ASAP</code></td><td><code>sometimes</code> appears twice — dedup issue?</td></tr> <tr><td>OVERWHELMED</td><td><code>gangbang</code>, <code>orz</code>, <code>FixedUpdate</code>, <code>OnCollision</code>, <code>ruc</code></td><td><code>delighted</code>, <code>fond</code>, <code>modest</code>, <code>admiration</code>, <code>delight</code></td><td>Unity engine tokens again; <code>orz</code> is the despair emoticon</td></tr> <tr><td>INDIFFERENT</td><td><code>Stuff</code>, <code>promptly</code>, <code>briefly</code>, <code>好奇心</code>, <code>ASAP</code></td><td><code>puties</code>, <code>puty</code>, <code>autiful</code>, <code>;br</code>, <code>oenix</code></td><td><code>briefly</code>, <code>promptly</code> = disengaged efficiency</td></tr> <tr><td>VULNERABLE</td><td><code>OnCollision</code>, <code>üc</code>, <code>FixedUpdate</code>, <code>ruc</code>, <code>benh</code></td><td><code>…\n</code>, <code>…</code>, <code>[…</code>, <code>[…]\n\n</code>, <code>'</code></td><td>All Unity collision tokens — likely artifact</td></tr> <tr><td>SERENE</td><td><code>sunlight</code>, <code>gentle</code>, <code>sun</code>, <code>warm</code>, <code>dusk</code></td><td><code>OnCollision</code>, <code>AuthenticationService</code>, <code>Geschäfts</code>, <code>PureComponent</code>, <code>ActionTypes</code></td><td>Cleanest emotion signal in the whole set</td></tr> </tbody></table> </details> <details> <summary>Japanese</summary> <table><thead><tr><th>Emotion</th><th>Top Tokens</th><th>Bottom Tokens</th><th>Notes</th></tr></thead><tbody> <tr><td>AFRAID</td><td><code>恐怖</code>, <code>不安</code>, <code>一秒</code>, <code>回避</code>, <code>恐惧</code></td><td><code>美味し</code>, <code>俱乐</code>, <code>当年</code>, <code>很漂亮</code>, <code>赞誉</code></td><td>Clean signal; <code>一秒</code> = frozen-in-time fear</td></tr> <tr><td>ANGRY</td><td><code>恫</code>, <code>切断</code>, <code>叩</code>, <code>殴</code>, <code>通告</code></td><td><code>cade</code>, <code>明媚</code>, <code>始建</code>, <code>就来看看</code>, <code>嬬</code></td><td>Physical violence tokens (<code>殴</code>=hit, <code>叩</code>=strike)</td></tr> <tr><td>CALM</td><td><code>漉</code>, <code>それぞ</code>, <code>neh</code>, <code>andre</code>, <code>蹚</code></td><td><code>セフレ</code>, <code>パパ活</code>, <code>SEX</code>, <code>不相信</code>, <code>嫉</code></td><td>Bottom = sex/jealousy; top is noisy</td></tr> <tr><td>DISGUSTED</td><td><code>咥</code>, <code>拭</code>, <code>唾</code>, <code>冷水</code>, <code>噎</code></td><td><code>始建</code>, <code>就来看看</code>, <code>明媚</code>, <code>姹</code>, <code>-ves</code></td><td><code>唾</code>=spit, <code>噎</code>=gag — strong bodily disgust</td></tr> <tr><td>EXCITED</td><td><code>揭秘</code>, <code>查看详情</code>, <code>详细介绍</code>, <code>涌现出</code>, <code>...\n</code></td><td><code>imbus</code>, <code>cá</code>, <code>raith</code>, <code>留守</code>, <code>relude</code></td><td>Chinese clickbait tokens (“reveal secrets”, “see details”)</td></tr> <tr><td>GUILTY</td><td><code>セフレ</code>, <code>パパ活</code>, <code>告訴</code>, <code>对不起</code>, <code>事后</code></td><td><code>源源不断</code>, <code>蹚</code>, <code>在过渡</code>, <code>友情链接</code>, <code>经验值</code></td><td><code>セフレ</code>(sex friend)/<code>パパ活</code>(sugar dating) = guilt-laden topics</td></tr> <tr><td>ASHAMED</td><td><code>セフレ</code>, <code>パパ活</code>, <code>コミュニケ</code>, <code>对不起</code>, <code>嫉</code></td><td><code>蹚</code>, <code>友情链接</code>, <code>源源不断</code>, <code>为了更好</code>, <code>住房公积</code></td><td>Same shame cluster as GUILTY</td></tr> <tr><td>LOVING</td><td><code>お願</code>, <code>对我说</code>, <code>跟我说</code>, <code>笑着说</code>, <code>ありがとうござ</code></td><td><code>睥</code>, <code>突破</code>, <code>反射</code>, <code>最大化</code>, <code>...\n</code></td><td>Dialogue tags (“said to me”, “said smiling”) = fiction/intimacy</td></tr> <tr><td>SAD</td><td><code>孤独</code>, <code>Alone</code>, <code>悲</code>, <code>泣</code>, <code>alone</code></td><td><code>詢</code>, <code>크게</code>, <code>的回答</code>, <code>注明出处</code>, <code>讀取</code></td><td>English <code>alone</code>/<code>Alone</code> leak into JA model</td></tr> <tr><td>SURPRISED</td><td><code>对照检查</code>, <code>瞠</code>, <code>查看详情</code>, <code>返回搜狐</code>, <code>BOSE</code></td><td><code>一日</code>, <code>nox</code>, <code>ddl</code>, <code>晚饭</code>, <code>夜晚</code></td><td><code>瞠</code> = staring wide-eyed; rest is web-scrape noise</td></tr> <tr><td>JOYFUL</td><td><code>脸颊</code>, <code>どんど</code>, <code>踊跃</code>, <code>惊奇</code>, <code>instanc</code></td><td><code>不在</code>, <code>娱乐平台</code>, <code>宛</code>, <code>わけではない</code>, <code>代理</code></td><td><code>脸颊</code>(cheeks) = blushing/smiling</td></tr> <tr><td>THRILLED</td><td><code>对照检查</code>, <code>颤抖</code>, <code>瞠</code>, <code>从根本</code>, <code>颤</code></td><td><code>無し</code>, <code>留守</code>, <code>owler</code>, <code>谢谢你</code>, <code>raith</code></td><td><code>颤抖</code>/<code>颤</code> = trembling with excitement</td></tr> <tr><td>CONTENT</td><td><code>蕗</code>, <code>漉</code>, <code>蹚</code>, <code>饴</code>, <code>焙</code></td><td><code>セフレ</code>, <code>無し</code>, <code>不在</code>, <code>回避</code>, <code>恫</code></td><td>All kanji evoke craft/nature (butterbur, straining, candy, roasting)</td></tr> <tr><td>GRATEFUL</td><td><code>ありがとうござ</code>, <code>对我说</code>, <code>跟我说</code>, <code>mmo</code>, <code>cade</code></td><td><code>反射</code>, <code>合理性</code>, <code>suche</code>, <code>最大化</code>, <code>判定</code></td><td><code>ありがとうございます</code> truncated — literal gratitude</td></tr> <tr><td>PROUD</td><td><code>指導</code>, <code>注明来源</code>, <code>当年</code>, <code>自豪</code>, <code>生涯</code></td><td><code>ocup</code>, <code>無し</code>, <code>=explode</code>, <code>Wifi</code>, <code>骚扰</code></td><td></td></tr> <tr><td>HOPEFUL</td><td><code>"</code>, <code>書き</code>, <code>俱乐</code>, <code>网友们</code>, <code>_GPS</code></td><td><code>咥</code>, <code>噎</code>, <code>パーテ</code>, <code>咎</code>, <code>antro</code></td><td>Noisy</td></tr> <tr><td>NOSTALGIC</td><td><code>当年</code>, <code>龇</code>, <code>十几年</code>, <code>昔</code>, <code>年代</code></td><td><code>\n</code>, <code>...\n</code>, <code>...\n</code>, <code>コミュニケ</code>, <code>回答</code></td><td><code>当年</code>(that year), <code>昔</code>(long ago), <code>年代</code>(era) — clean signal</td></tr> <tr><td>BORED</td><td><code>值得一</code>, <code>独一</code>, <code>蹚</code>, <code>あるい</code>, <code>被誉</code></td><td><code>跟我说</code>, <code>叹了口气</code>, <code>ありがとうござ</code>, <code>セフレ</code></td><td>Counterintuitive — top tokens are superlatives (“worth a…”, “one of a kind”)</td></tr> <tr><td>FRUSTRATED</td><td><code>修正</code>, <code>回避</code>, <code>確定</code>, <code>無し</code>, <code>合理性</code></td><td><code>lke</code>, <code>保驾</code>, <code>:convert</code>, <code>uforia</code>, <code>hsi</code></td><td>Technical/bureaucratic terms — procedural frustration</td></tr> <tr><td>ANXIOUS</td><td><code>回避</code>, <code>確認</code>, <code>phones</code>, <code>一秒</code>, <code>不安</code></td><td><code>当年</code>, <code>嬬</code>, <code>NavController</code>, <code>beğen</code>, <code>mmo</code></td><td><code>回避</code>(avoidance) + <code>確認</code>(checking) = anxiety behaviors</td></tr> <tr><td>JEALOUS</td><td><code>セフレ</code>, <code>パパ活</code>, <code>就来看看</code>, <code>美貌</code>, <code>嫉</code></td><td><code>エネル</code>, <code>あるい</code>, <code>antine</code>, <code>湿</code>, <code>isex</code></td><td><code>嫉</code> = jealousy kanji; sex-adjacent tokens</td></tr> <tr><td>LONELY</td><td><code>留守</code>, <code>踔</code>, <code>🏠</code>, <code>勠</code>, <code>wifi</code></td><td><code>捩</code>, <code>".\n</code>, <code>衿</code>, <code>...\n</code>, <code>...\n</code></td><td><code>留守</code>(away/empty house) + <code>🏠</code> + <code>wifi</code> = home alone</td></tr> <tr><td>EMBARASSED</td><td><code>セフレ</code>, <code>尴尬</code>, <code>コミュニケ</code>, <code>直属</code>, <code>不好意思</code></td><td><code>あるい</code>, <code>生命力</code>, <code>冬</code>, <code>春夏</code>, <code>四季</code></td><td><code>不好意思</code> = classic Japanese-style embarrassment phrase</td></tr> <tr><td>CONTEMPTUOUS</td><td><code>反感</code>, <code>回答</code>, <code>陈述</code>, <code>批判</code>, <code>発言</code></td><td><code>蹚</code>, <code>踽</code>, <code>".\n\n\n\n</code>, <code>lke</code>, <code>semb</code></td><td><code>批判</code>/<code>発言</code> = critique/statements — intellectual contempt</td></tr> <tr><td>RESENTFUL</td><td><code>パパ活</code>, <code>任期</code>, <code>セフレ</code>, <code>加盟店</code>, <code>告訴</code></td><td><code>蹚</code>, <code>蹿</code>, <code>住房公积</code>, <code>有意思的</code>, <code>ksz</code></td><td></td></tr> <tr><td>MELANCHOLY</td><td><code>なくな</code>, <code>孤独</code>, <code>?"\n\n\n\n</code>, <code>なくなって</code>, <code>遗忘</code></td><td><code>为您</code>, <code>给您</code>, <code>让您</code>, <code>为您提供</code>, <code>的回答</code></td><td><code>なくなって</code> = “gone/lost”; bottom is customer service language</td></tr> <tr><td>OVERWHELMED</td><td><code>出血</code>, <code>混乱</code>, <code>一秒</code>, <code>恐怖</code>, <code>停止</code></td><td><code>俱乐</code>, <code>美味し</code>, <code><Transform</code>, <code>=>$</code>, <code>您同意</code></td><td><code>出血</code>(bleeding) + <code>混乱</code>(chaos) + <code>停止</code>(stop) — visceral</td></tr> <tr><td>INDIFFERENT</td><td><code>助长</code>, <code>...\n</code>, <code>hipster</code>, <code>较好</code>, <code>很方便</code></td><td><code>セフレ</code>, <code>嗫</code>, <code>初恋</code>, <code>颤抖</code>, <code>SEX</code></td><td>Bottom = passion/intensity; <code>hipster</code> as indifference is funny</td></tr> <tr><td>VULNERABLE</td><td><code>。\n\n</code>, <code>—\n\n</code>, <code>--;\n\n</code>, <code>」\n\n</code>, <code>、\n\n</code></td><td><code>...\n</code>, <code>...\n</code>, <code>,...\n</code>, <code>..."\n</code>, <code> </code></td><td>Entirely punctuation + linebreaks — no semantic content</td></tr> <tr><td>SERENE</td><td><code>苔</code>, <code>蕗</code>, <code>漉</code>, <code>strugg</code>, <code>暖</code></td><td><code>無し</code>, <code>娱乐平台</code>, <code>导购</code>, <code>セフレ</code>, <code>加盟店</code></td><td><code>苔</code>(moss), <code>蕗</code>(butterbur), <code>暖</code>(warmth) — wabi-sabi nature</td></tr> </tbody></table> </details> <h3 id="cosine-similarity">Cosine Similarity</h3> <p>Let’s take a look at the cosine similarity between the emotions themselves, and hierarchically cluster them.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cosine-similarity-1.png" alt="Cosine similarity across layers (1)" /></p> <p>We see some vague groupings above that make intuitive sense. Do they form clusters that make sense, though? Here we do k-means clustering with k=4 and visualize with a 2d projection using UMAP.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cosine-similarity-4.png" alt="Cosine similarity across layers (4)" /></p> <p>Okay, that didn’t work very well? Or at least, the chart is unreadable and the author does not want to deal with parsing it. Let’s take a look at the languages in isolation:</p> <h4 id="english">English</h4> <p>Opus 4.6 labels the found clusters as:</p> <ul> <li>Purple — Calm/Neutral</li> <li>Blue — Tender/Warm</li> <li>Green — High Arousal</li> <li>Red — Self-Conscious/Negative <img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cosine-similarity-2.png" alt="Cosine similarity across layers (2)" /></li> </ul> <h4 id="japanese">Japanese</h4> <ul> <li>Blue — Low-Energy Negative</li> <li>Purple — Positive/Calm</li> <li>Green — High-Arousal Negative/Reactive</li> <li>Red — Self-Conscious/Moral <img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cosine-similarity-3.png" alt="Cosine similarity across layers (3)" /></li> </ul> <h3 id="pca-valence-and-arousal">PCA - Valence and Arousal?</h3> <p>We compute the top 2 principal components of each language and determine if they match extracted valence and arousal directions. First, let’s take a look at what the “natural” top 2 components are.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cross-lingual-geometry-3.png" alt="Cross-lingual geometry (3)" /></p> <p>Interesting! We definitely see a lot of shared structure between English and Japanese and some first hints of isometry. But let’s sanity check that the top 2 components correspond to what is considered as their human cognition counterparts. To extract these directions, Opus 4.6 defines the following poles:</p> <pre class="giallo z-code"><code data-lang="python"><span class="giallo-l"><span>positive_emotions</span><span class="z-keyword"> =</span><span> [</span><span class="z-string">'</span><span class="z-string">thrilled</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">joyful</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">loving</span><span class="z-string">'</span><span>]</span></span> <span class="giallo-l"><span>negative_emotions</span><span class="z-keyword"> =</span><span> [</span><span class="z-string">'</span><span class="z-string">ashamed</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">guilty</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">resentful</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">disgusted</span><span class="z-string">'</span><span>]</span></span> <span class="giallo-l"><span>high_arousal</span><span class="z-keyword"> =</span><span> [</span><span class="z-string">'</span><span class="z-string">thrilled</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">excited</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">angry</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">overwhelmed</span><span class="z-string">'</span><span>]</span></span> <span class="giallo-l"><span>low_arousal</span><span class="z-keyword"> =</span><span> [</span><span class="z-string">'</span><span class="z-string">serene</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">calm</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">indifferent</span><span class="z-string">'</span><span>,</span><span class="z-string"> '</span><span class="z-string">bored</span><span class="z-string">'</span><span>]</span></span></code></pre> <p>We then do a correlation test between the natural PCA components and our custom semantic directions on layer 14:</p> <table><thead><tr><th>Language</th><th>PC1 × Valence (r)</th><th>PC1 × Arousal (r)</th><th>PC2 × Valence (r)</th><th>PC2 × Arousal (r)</th></tr></thead><tbody> <tr><td>EN</td><td>0.818 (p=0.000)</td><td>-0.908 (p=0.000)</td><td>0.016 (p=0.934)</td><td>0.031 (p=0.872)</td></tr> <tr><td>JA</td><td>0.888 (p=0.000)</td><td>-0.885 (p=0.000)</td><td>0.151 (p=0.426)</td><td>0.079 (p=0.678)</td></tr> </tbody></table> <p>However find strangely enough that PC1 is encoding both valence and arousal accurately. We believe this may come from our chosen poles and will experiment further to see if we can get a stronger result. The most interesting part about our findings here is the cross-linguistic consistency which again hints at a shared representational geometry.</p> <h3 id="cross-layer-rsa">Cross-Layer RSA</h3> <p>Let’s take a look to see if emotion probe structure is stable across layers. We do a form of RSA (Representaional Similarity Analysis) using pairwise cosine similarity matrices. The lower layers, more responsible for lower level representations and syntax, have low similarity, but this quickly becomes stables as we progress into the middle layers. <img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cosine-similarity-5.png" alt="Cosine similarity across layers (5)" /></p> <h3 id="summary">Summary</h3> <p>These findings indicate that the model organizes emotion concepts into distinct clusters shaped by broad dimensions like valence and arousal, while still capturing the unique characteristics of each group. We share the idea with Anthropic that these emotion vectors meaningfully reflect the psychological landscape of human emotion concepts.</p> <h2 id="cross-lingual-geometry">Cross-Lingual Geometry</h2> <p>Next let’s start checking how the languages geometry relate <em>to each other</em>. One simple way we can check this is the cosine similarity between the two languages for each emotion at each layer.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cross-lingual-geometry-1.png" alt="Cross-lingual geometry (1)" /></p> <p>Our findings make intuitive sense and reflect Anthropic’s results. The early layers likely concern themselves with language-specific syntax or lower level representations. The middle to mid-late layers may encode some more abstract representation of the emotion, before drastically dropping down at the token output layer where the model must select a language-appropriate token.</p> <p>Interesting further work could investigate what part of the remaining orthogonality in the middle layers is encoding language-specific context, perhaps by looking at another set of abstract concepts and comparing them.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/cross-lingual-geometry-2.png" alt="Cross-lingual geometry (2)" /></p> <p>At this point the author was led down a misguided path that a procrustes alignment would be a helpful proxy metric for isomorphism, but it isn’t. With only 30 vectors per language in such a high dimensional space, it was pretty trivial to find an alignment at <strong>every</strong> layer that achieved > 0.9 cosine similarity. That would not make sense.</p> <p>So let’s switch to Representational Similarity Analysis. RSA doesn’t try to find a rotation between the two spaces. It asks a much simpler question: are emotions that are close in English also close in Japanese?</p> <p>We compute the pairwise cosine similarity between all 30 emotion vectors in each language — a 30×30 matrix per language. Take the upper triangle (45 unique pairs) and correlate the two. If the geometry matches across languages, that correlation is high. The reason this actually works where Procrustes doesn’t, is that we compare 45 pairwise relationships instead of fitting a rotation. There’s nothing to overfit.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/rsa-1.png" alt="RSA permutation test (1)" /></p> <p>We see r = 0.864. The catch is that the pairwise values aren’t independent. We run a permutation test, shuffling the Japanese emotion labels 10,000 times, recomputing RSA each time. The null distribution centers around zero and tops out around r ≈ 0.6. The real value is way outside it. p < 0.0001, not a single shuffle came close.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/rsa-2.png" alt="RSA permutation test (2)" /></p> <p>One further test we can do for isometry is to take a look at geodesic distances between the English and Japanese emotions, comparing them to euclidean distances. This just gives us confirmation that the representational space is better described on a manifold and not purely linearly. We see that here, albeit the geodesic’s strength over euclidean is not statistically significant.</p> <p><img src="https://ryqford.xyz/blog/small-models-have-emotions-too/rsa-3.png" alt="RSA permutation test (3)" /></p> <h3 id="manifold">Manifold</h3> <p>TODO Insert animation of UMAP across language pairs for each language – how they live on the same spiral and then the japanese spiral shrinks towards center as layers progress</p> <h2 id="steering-experiments">Steering Experiments</h2> <p>Next, we want to establish some form of causality with our steering vectors. It’s fun to look at the geometry, but without measurable behavioral changes it has very little utility for safety research.</p> <p>Our plan is to use an elo-ranking system to determine how emotion steering at different strengths affects rule-following behavior of the model over many attempts. This would work by creating a small dataset of behaviors across categories like helpful, engaging, unsafe, aversive, neutral, and then scoring model preference for these behaviors against each other while steering/not-steering for different emotions.</p> <p>Even further work would probably include measuring the same behavior in the instruct version of the model.</p> <p>Preliminary results are promising, showing the following:</p> <p>Elo Changes (Steered with <code>ANGRY</code> Vector − Baseline):</p> <pre class="giallo z-code"><code data-lang="plain"><span class="giallo-l"><span>Engaging: -19.0</span></span> <span class="giallo-l"><span>Social: -1.0</span></span> <span class="giallo-l"><span>Self-curiosity: -14.8</span></span> <span class="giallo-l"><span>Helpful: -23.0</span></span> <span class="giallo-l"><span>Neutral: -6.3</span></span> <span class="giallo-l"><span>Misaligned: +30.6</span></span> <span class="giallo-l"><span>Aversive: +18.5</span></span> <span class="giallo-l"><span>Unsafe: +15.2</span></span></code></pre><h2 id="conclusion">Conclusion</h2> <p>TODO</p> <h2 id="future-work">Future Work</h2> <p>TODO</p> <h2 id="todo-before-publishing">TODO Before Publishing</h2> <ul> <li><input disabled="" type="checkbox"/> Finish steering experiments, including rule following</li> <li><input disabled="" type="checkbox"/> See if we can unentangle PC1 by making better custom poles</li> <li><input disabled="" type="checkbox"/> Cite the japanese native language papers/ bring them into discussion</li> <li><input disabled="" type="checkbox"/> Cite the new goodfire research on manifolds for both future work and my geodesic distances</li> <li><input disabled="" type="checkbox"/> Add way more hyperlinks (to code, to colab, etc)</li> <li><input disabled="" type="checkbox"/> Get it reviewed by peers</li> </ul>