Bookmarks (682)

Published on May 17, 2024 5:30 PM GMTDeepMind's RSP is here. Excerpt from the blogpost:Today, we are introducing our Frontier Safety Framework - a set of protocols for proactively identifying future AI capabilities that could cause severe harm and putting in place mechanisms to detect and mitigate them. Our Framework focuses on severe risks resulting from powerful capabilities at the model level, such as...

Published on May 17, 2024 4:25 PM GMTA short summary of the paper is presented below.This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) .TL;DR: We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the...

Published on May 17, 2024 3:57 PM GMTThere are many resources for those who wish to contribute to AI Safety, such as courses, communities, projects, jobs, events and training programs, funders and organizations. However, we often hear from people that they have trouble finding the right resources. To address this, we've built AISafety.com as a central hub—a list-of-lists—where community members maintain and curate these...

Published on May 17, 2024 9:28 AM GMTEpistemic Status: I thought about and wrote each paragraph in 10 minutes total, with slight editing afterwards.I hope I'm not too late to the party! I wrote this up quite a few months ago and found that I delayed indefinitely editing it before publication. I decided it's probably best to post a not-maximally-edited version of my final exam. This...

Published on May 17, 2024 11:56 AM GMTA couple of weeks ago three European economists published this paper studying the female income penalty after childbirth. The surprising headline result: there is no penalty.Setting and MethodologyThe paper uses Danish data that tracks IVF treatments as well as a bunch of demographic factors and economic outcomes over 25 years. Lundborg et al identify the causal effect...

Published on May 17, 2024 1:20 AM GMTI expect it would be useful when developing an understanding of the language used on LW.Discuss

Published on May 17, 2024 12:25 AM GMTThis is a D&D.Sci scenario: a puzzle where players are given a dataset to analyze and an objective to pursue using information from that dataset.Duke Arado’s obsession with physics-defying architecture has caused him to run into a small problem. His problem is not – he affirms – that his interest has in any way waned: the menagerie...

Published on May 16, 2024 10:21 PM GMTI keep seeing this meme doing the rounds where people present ChatGPT with a common logic problem or riddle, only with some key component changed to make it trivial. ChatGPT has seen the original version a million times, so it gives the answer to the original, not the actually correct and obvious answer.The idea is to show...

Published on May 16, 2024 8:47 PM GMTSharing my detailed walk-through on using the UK AI Safety Institute's new open source package Inspect for AI evals.Main points:Package released in early May 2024 is here: https://github.com/UKGovernmentBEIS/inspect_aiSeems easy to use and removes boiler-plate code. I am new to evals so I do not know what experienced researchers would look for in such a tool. I am...

Published on May 16, 2024 6:56 PM GMTContra this post from the SequencesIn Eliezer's sequence post, he makes the following (excellent) point:I can’t find any theorem of probability theory which proves that I should appear ice-cold and expressionless.This debunks the then-widely-held view that rationality is counter to emotions. He then goes on to claim that emotions have the same epistemic status as the beliefs...

Published on May 15, 2024 6:33 PM GMTGPT-4o both has a new tokenizer and was trained directly on audio (whereas my understanding is that GPT-4 was trained only on text and images). Is there precedent for upgrading a model to a new tokenizer? It seems like it's probably better to think of it as an entirely new model. If that's the case, what actually...

Published on May 15, 2024 6:03 PM GMTI suspect some number of LWers have taken or are are considering using Lumina's probiotic. If you're in either of those camps, Klee's post might be worth reading. He paints a picture of an unprofessional company skirting regulations and risking customers health to sell a dubious health product. I can't speak to the veracity of those claims,...

Published on May 15, 2024 5:30 PM GMTThis is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?SummaryRising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.Public OpinionOne hypothesis is that partisanship was useful, or even necessary, for...

Published on May 15, 2024 12:52 PM GMTThis question is not very well-posed, but I've done my best to make it as well-posed as I can.Suppose that humans with sufficiently functional brains are able have subjective experiences that transcend the "easy problems of consciousness".I'm interested in understanding if this can be reasonably accepted without also concluding a theory of some sort of "panpsychism". For...

Published on May 15, 2024 11:57 AM GMTCrossposted from my blog: https://invertedpassion.com/how-to-be-a-messy-thinker/I love thinking about thinking. Give me a research paper on rationality, cognitive biases or mental models, and I’ll gobble it up. Given the amount of knowledge I’ve ingested on these topics, I had always assumed that I’m a clear thinker.Recently, though, it hit me like a lightning strike that this belief is...

Published on May 15, 2024 2:50 AM GMT A few years ago I ported my whistle synth system from my laptop to a Raspberry Pi. This was a big improvement, but I still wasn't that happy: To get good quality audio in and out I was using a 2i2 audio interface, which is expensive, bulky, and has a lot of buttons and knobs that...

Published on May 15, 2024 12:58 AM GMTTLDR: In the last two posts, we showed that optimizing for a proxy can fail to increase true utility, but only when the error is heavy-tailed. We now show that this also happens in RLHF with a KL penalty.This post builds on our earlier result with a more realistic setting and assumptions:Rather than modeling optimization as conditioning...

Published on May 15, 2024 12:45 AM GMTIlya Sutskever and Jan Leike have resigned. They led OpenAI's alignment work. Superalignment will now be led by John Schulman, it seems. Jakub Pachocki replaced Sutskever as Chief Scientist.Reasons are unclear (as usual when safety people leave OpenAI).The NYT piece and others I've seen don't really have details. Archive of NYT if you want to read it...

Published on May 15, 2024 12:20 AM GMTI've been told that my number of blog posts is impressive, but my personal notes are much larger than my blog, over a million words and with higher information density. Since I've had a bit of practice taking notes, I thought I'd describe the system I developed. It's more complex than some integrated solutions, but it's powerful,...

Published on May 15, 2024 12:13 AM GMTMIRI updates:MIRI is shutting down the Visible Thoughts Project.We originally announced the project in November of 2021. At the time we were hoping we could build a new type of data set for training models to exhibit more of their inner workings. MIRI leadership is pessimistic about humanity’s ability to solve the alignment problem in time, but this...

Published on May 13, 2024 6:33 PM GMTOpenAI just announced an improved LLM called GPT-4o.From their websiteToday, GPT-4o is much better than any existing model at understanding and discussing the images you share. For example, you can now take a picture of a menu in a different language and talk to GPT-4o to translate it, learn about the food's history and significance, and get...

Published on May 13, 2024 5:20 PM GMT This Saturday was Porchfest in Somerville, an annual festival where musicians around the city play on their porches and people wander around listening. As in the past few years Cecilia and I ( Kingfisher) played for contra dancing: Harris Lapiroff called: If anyone has pictures of videos from the set, I'd love to see them as...

Published on May 13, 2024 5:17 PM GMTThis article is the first in a series I plan to publish on different aspects of AI Safety group strategy. The aim is that, eventually, these articles will form the basis for a new resource center for AI Safety Groups. Note that these articles aren’t being published in any particular order.TL;DR: AI safety groups should carefully consider their branding strategy to...

Published on May 13, 2024 2:55 PM GMTA stance against student debt cancellation doesn’t rely on the assumptions of any single ideology. Strong cases against student debt cancellation can be made based on the fundamental values of any section of the political compass. In no particular order, here are some arguments against student debt cancellation from the perspectives of many disparate ideologies.Equity and FairnessStudent...

Published on May 13, 2024 12:30 PM GMTAs I note in the third section, I will be attending LessOnline at month’s end at Lighthaven in Berkeley. If that is your kind of event, then consider going, and buy your ticket today before prices go up. This month’s edition was an opportunity to finish off some things that got left out of previous editions or...

Published on May 13, 2024 9:18 AM GMTWhat are the best ways to figure out if someone who contacted you / a piece of text / an image is AI generated?Discuss

Published on May 13, 2024 8:52 AM GMTAuthor’s Note: Though I’m currently a governance researcher at Convergence Analysis, this post is unaffiliated with Convergence. The opinions expressed are solely my own.You’ve seen it a dozen times at this point. You’re probably broadly aligned philosophically, but haven’t thought terribly deeply about the details. You generally support Andrew Yang’s $12k / year “Freedom Dividend” as “moving...

Published on May 13, 2024 7:53 AM GMTOn AI and Jobs: How to Make AI Work With Us, Not Against Us With Daron AcemogluHere is Claude.ai's summary of Daron Acemoglu's main ideas from the podcast:Historically, major productivity improvements from new technologies haven't always translated into benefits for workers. It depends on how the technologies are used and who controls them.There are concerns that AI...

Published on May 13, 2024 6:06 AM GMTI'm looking for personal rules one might live by which adhere to a specific criteria outlined below, following an example. I have a personal rule I've been following which is "No looking at screens in the bed where I sleep" I find this to be an extremely helpful and successful rule despite being someone who struggles to...

Published on May 12, 2024 9:58 PM GMTmostly written by Evan MiyazonoMinimum viable introductionWe ran a workshop on Mathematical Boundaries from April 10-14. This was the successor of the Conceptual Boundaries Workshop (retrospective, last update) Feb 10-12The overlap in participants and approach was fairly low (notably lower than intended, due to availability restricting participation, which in turn led to a natural difference in approach)Intent:The first...

Published on May 11, 2024 6:30 PM GMTI'm doing some initial investigation for a Notes on Virtues sequence write-up about poise & gracefulness (possibly touching also on related things like confidence, composure, cool, gravitas, command, bearing, etc.) seen as a virtue (a characteristic habit that promotes or exhibits human flourishing).Can you recommend some resources (e.g. books, papers, interviews) I would be wise to review...

Published on May 11, 2024 6:18 PM GMTMarcus Hutter and his PhD students David Quarel and Elliot Catt have just published a new textbook called An Introduction to Universal Artificial Intelligence."Universal AI" refers to the body of theory surrounding Hutter's AIXI, which is a model of ideal agency combining Solomonoff induction and reinforcement learning. Hutter has previously published a book-length exposition of AIXI in...

Published on May 11, 2024 1:00 PM GMTIt is easier to ask than to answer. That’s my whole point.It is much cheaper to ask questions than answer them so beware of situations where it is implied that asking and answering are equal. Here are some examples:Let's say there is a maths game. I get a minute to ask questions. You get a minute to answer them....

Published on May 11, 2024 9:31 AM GMTI've been on the lookout for new jobs recently and one thing I have noticed is that the market seems flooded with ads for AI-related jobs. What I mean is not work on building models (or aligning them, alas), but rather, work on building applications using generative AI or other advances to make new software products. My...

Published on May 11, 2024 5:17 AM GMTTo some, it might seem like a strange question. If you think of being college-educated as a marker of class (or personhood), the fact that I don't have a degree at age of thirty-six (!!) probably looks like a scandalous anomaly, which it would be only natural for me to want to remediate at the earliest opportunity....

Published on May 11, 2024 2:40 AM GMT When I play live I have a bunch of instruments, including: Mandolin: an electric mandolin Computer: a custom MIDI mapper driven by keyboard, foot drums, and breath controller. Bass whistle: a whistle-controlled bass synthesizer I also have some effects, primarily a talkbox and an audio-to-audio synth pedal. Normally I route the mandolin into the effects, but...

Published on May 11, 2024 12:09 AM GMTCo-Authors: @Rocket, @Ryan Kidd, @LauraVaughan, @McKennaFitzgerald, @Christian Smith, @Juan Gil, @Henry SleightThe ML Alignment & Theory Scholars program (MATS) is an education and research mentorship program for researchers entering the field of AI safety. This winter, we held the fifth iteration of the MATS program, in which 63 scholars received mentorship from 20 research mentors. In this post, we...

Published on May 11, 2024 12:08 AM GMTTLDR; I demonstrate the use of refusal vector ablation on Llama 3 70B to create a bad agent that can attempt malicious tasks such as trying to persuade and pay me to assassinate another individual. I introduce some early work on a benchmark for Safe Agents which comprises two small datasets, one benign, one bad. In general, Llama 3...

Published on May 10, 2024 6:34 PM GMTOne of the fun things to do when learning first order logic is to consider how the meaning of propositions dramatically changes based on small switches in the syntax. This is in contrast to natural language, where the meaning of a phrase can be ambiguous and we naturally use context clues to determine the correct interperation.An example...

Published on May 10, 2024 6:34 PM GMTThe following is the first in a 6 part series about humanity's own alignment problem, one we need to solve, first.What Is Alignment?ALIGNMENT OF INTERESTSWhen I began exploring non-zero-sum games, I soon discovered that achieving win-win scenarios in the real world is essentially about one thing - the alignment of interests.If you and I both want the...

Published on May 9, 2024 5:33 PM GMTIntroductionThe full draft textbook is available here.Epistemic Status: I'm pretty satisfied with this document. I wrote it because it doesn't seem like we've made any major breakthroughs in alignment in the last year, and I wanted to consolidate what I know. And beyond alignment, it seems to me that a large class of strategies are quite important...

Published on May 9, 2024 3:43 PM GMTPredicting the future is hard, so it’s no surprise that we occasionally miss important developments.However, several times recently, in the contexts of Covid forecasting and AI progress, I noticed that I missed some crucial feature of a development I was interested in getting right, and it felt to me like I could’ve seen it coming if only I...

Published on May 9, 2024 2:50 PM GMT Somerville historically had a zoning ordinance limiting housing units to at most four unrelated people: any number of persons related by blood, marriage, adoption, or foster care agreement and up to three (3) additional unrelated persons living together as a single housekeeping unit This is something I'd been unhappy about for years, and was enthusiastic about...

Published on May 9, 2024 2:20 PM GMTIt was a remarkably quiet announcement. We now have Alpha Fold 3, it does a much improved job predicting all of life’s molecules and their interactions. It feels like everyone including me then shrugged and went back to thinking about other things. No cool new toy for most of us to personally play with, no existential risk...

Published on May 9, 2024 2:10 PM GMTOr rather Samuel Hammond does. Tyler Cowen finds it interesting but not his view. I put up a market, and then started looking. Click through to his post for the theses. I will be quoting a few of them in full, but not most of them. I am not trying to be exact with these probabilities when...

Published on May 9, 2024 12:07 PM GMTWhen it comes to mesa optimization, there’s usually two optimizers mentioned: the “base optimizer”, such as SGD, the process used for training the model. And the “mesa optimizer”, meaning roughly “the optimizer beneath the optimizer”. One key question in this context is when and under which circumstances such mesa optimizers may occur. This question is relevant for...

Published on May 9, 2024 6:40 AM GMTTLDRWe develop a technique to try and detect if a NN is doing planning internally. We apply the decoder to the intermediate representations of the network to see if it’s representing the states it’s planning through internally. We successfully reveal intermediate states in a simple Game of Life model, but find no evidence of planning in an...

Published on May 9, 2024 12:49 AM GMTThis is the first post in a little series I'm slowly writing on how I see forecasting, particularly conditional forecasting; what it's good for; and whether we should expect people to agree if they just talk to each other enough.Views are my own. I work at the Forecasting Research Institute (FRI), I forecast with the Samotsvety group, and...

Published on May 8, 2024 11:17 PM GMTI mostly wrote this for facebook, but it ended up being a whole-ass post so I figured I'd put it here too. I'm helping run "LessOnline: A Festival of Writers Who Are Wrong On the Internet (But Striving To Be Less So)".I'm incentivized to say nice things about the event. So, grain of salt and all. But, some...

Published on May 8, 2024 7:09 PM GMTThis is the tenth post in my series on Anthropics. The previous one is Beauty and the Bets.IntroductionIn my previous posts I've been talking about the actual object-level disagreement between halfers and thirders - which of the answers formally is correct and which is not. I've shown that there is one correct model for the Sleeping Beauty...