llm/38bf93a2-b4fa-4fa5-b786-816deb71dcbf/92c4ce14-52e4-4719-8b1a-8287937f2a30-input.json
Summarize the following content from a Notion page and its linked resources in approximately 500 words. Focus on the main ideas, key points, and important information. Write in a clear, direct style. Format your output in Markdown.
<content>
=== Make a Rough Draft of Phase 2 (https://www.notion.so/Make-a-Rough-Draft-of-Phase-2-2a01a29f178980b6a8e1de4ab322f035) === Work from AI Watersheds Phase 2: Indicators [Child Page: Steve’s Notes from the Phase 2 Brainstorming Doc] Brainstorming doc Why Do Measurements? Thoughts on the purpose of forecasting AI’s schedule General trend visibility for several years (important for lots of policy + other questions) At least several years notice of transformational change Estimate likely steepness & extent of transformational change Ask panel for feedback on this Josh: interested in the connection between current policy questions and our questions here. Would be great to get more specific / explicit about the connections with policy questions. Josh: interested in the connection between current policy questions and our questions here. Would be great to get more specific / explicit about the connections with policy questions. General thoughts on what to measure Practical utility is a function of both AI model capability and complementary app innovation (coding models vs. Cursor) Approaches to measurement Josh (+’d by Helen, Nikola, Sayash): Noting cross-cutting theme: Generally seems like more high-quality surveys of companies would be valuable. A lot of ideas involve sensitive data, so would require controlled access (and/or governmental involvement?) Using AIs to qualitatively grade lots of screen recordings and other messy high-volume data Find ways of calibrating / correlating benchmark scores with measurements we care about more directly? (METR time horizons being an example in this direction) Generally use lots of different measurements and try to map them to common units for cross-checking Surveys / prediction markets; focus real-world measurements (e.g. uplift studies) on areas of disagreement Or get both sides of a dispute to agree on a cheaper proxy measurement Case studies How firms in various sectors are using AI; scale of tasks assigned to AI / nature + granularity of interaction with humans; degree of trust / freedom of action given to AI Deep study of revenue / spending / profit across the AI value chain (and by domain) Also: number of “complementary innovation” (Sayash’s term) AI-powered apps, adoption rate, # of customers, “Surveys of customers to see how they are improving their workflows”, etc. Helen: something about revenue from AI more broadly vs. revenue of frontier model providers. What can we say about open models eating into OpenAI / Google / Anthropic revenue? Broad metrics (a la Epoch), e.g. electricity used by AI, global private investment Keep pushing benchmarks toward greater realism of both input (task specification + context) and output (not just “passes tests”, but “PR is mergeable” / other measures of quality) Nikola: need more realistic benchmarks. Source PRs (ideally from inside AI labs) and test AIs on them. Analysis of usage at the model level (Anthropic economic AI index) Logs of agent usage i.e. usage of model context protocol servers, pulls [Sayash] Analysis of AI logs from them being asked to solve various tasks (we’re doing this with 10B tokens of HAL agent log data, but could imagine increasing many fold) Social media is subject to transparency measures, AI companies currently are not. It’d be great to have controlled researcher access to e.g. ChatGPT transcripts. Lots of general data sets for public analysis AIs really like to email Eliezer, that’s an indicator of various types of strange AI behavior. Ajeya: there was an NYTimes article about a journalist getting similar emails. Could there be a “weird AI shit” incident tracker that would categorize reports + report trends? Josh: MIT AI incident repository? Populated from public reports – links to existing reports. Ajeya: hard to make sense of it. Needs to be a larger team sifting the signal from the noise + writing blog posts. Doesn’t track veracity. AI Risk repository / incidents database is relevant here: https://airisk.mit.edu/, also https://incidentdatabase.ai/ Ajeya comment: Too hard to make sense of current databases. Need someone to do better synthesis. Make an open source AI company where the company itself is open source. Have it try to use AI as much as possible, make all its logs about everything (Slack etc) public, study that. Sayash: there’s been a lot of work on the science of AI evaluations. We don’t know the shape of the DAG, how AI inputs translate into outputs. We could do more to distinguish between different DAGs – which is a core source of disagreement, for instance Sayash has a very different DAG in mind. Inputs might include things like data, compute; output measured in the real world. Econometrics. Josh: Meta measurements: Big increase in expected OpenAI revenue, according to a prediction market, based on a reported algorithmic breakthrough Pushback from Ryan+Ajeya: limitations of prediction markets mean it’ll be tough for them to pick this up reliably Surveys at frontier AI companies Surveys of the broader workforce Data center construction spend → currently dependent on media reporting, is there some way to get better/more consistent data on this? Trying to unpack inputs/outputs per Sayash’s point about DAGs/understanding how improvements in one element flow through to improvements in other elements Lots of (different types of) surveys Helen: important to track military applications (broadly – decision support, targeting, etc.). Important to track freedom of action here. Measuring Utility Metrics about fluid intelligence, messy tasks Benchmarks for domains vs. skills Domains: cyber/law/professional labor markets Skills: Context awareness, reliability, long-context reasoning, sample efficiency Need deep info for each sector before assessing results. I.e. persuasion surveys overestimate results because opinion change dissipates 6 weeks later. I like Steve’s note that AI can continually be used cheaply. Uplift RCTs in real-world settings, measuring real-world outcomes (ideally: profit!) e.g. if you get an actual large company to actually do randomized staged rollout of enterprise LLMs that’s great Ajeya: three-armed uplift trials, AI / human / cyborg. Josh: Persuasion: Cost for AI to swing a vote, relative to other leading interventions (Josh: lots of polysci studies here) AI-bio: % of amateurs that can synthesize flu Look for leapfrog companies that get on board with a new technology ahead of the incumbents, that could highlight utility ahead of adoption. e.g. fintech cos using AI in ways that let them overtake traditional banks, or healthcare startups using AI in ways that let you bypass doctors’ offices Power user case studies – screen recordings + deep interviews of people who (think they) are getting enormous uplift Organization version of this: find those YC startups that supposedly do 95% of everything with AI, study them (“power organizational users”) Field trials (a la AI Village) and realistic benchmarks Do a time horizon analysis (a la METR), but on real-world usage? Also, qualitative analysis of the size + difficulty + nature of tasks successfully delegated to AI (and dig into ranges of “successfully”) Building on Josh’s work test idea: could a METR-like org collect work test setups from a range of places (under NDA so as not to break the work tests) and centralize the work of eliciting good performance, figuring out measurement, reporting results, etc? Ryan: I think uplift will be messy (because of humans being messy and possibility for phase changes in how AI should be used productively), so I’m more into end-to-end automation of actual tasks. Also, we ultimately care most about full automation regimes and predicting when this will happen. Uplift studies correspondingly seem somewhat worse than looking at full automation of tasks of varying difficulty/size (and benchmark transfer to actual things people are doing in their job so we don’t need to constantly run these tests). Maybe you can get AI companies to run semi-informal experiments with uplift internally and publish results: many companies have effectively committed to doing this eventually due to their safety policies. I think Anthropic might open to this, but there are various sources of trickyness here. Early reports of AI doing remarkable things (e.g. major scientific discovery / insight; solving a Millennium Prize Problem in mathematics) pass@any for existing benchmarks can predict pass@1 in the future Helen: where is the ceiling on various capabilities (headroom)? This seems under-investigated. Sayash: are most tasks like chess, or like writing? (Ryan: unclear whether writing is the right contrast here.) Are there less controversial examples than writing? Taking out the trash: saturates. Helen: the writing example highlights we’re way under-theorized on this. Quality of parenting is another example. “Persuasion” is poorly specified. Abi: a scientific breakthrough requires an insight that goes against dogma. Could be considered an aspect of very high-quality writing. Ryan: I care most about questions like “will energy production 100x within 5 years of full AI R&D automation if people want this to happen”. Seems like this has many possible routes in terms of capabilities head room. Sayash: three broad categories: computational limits (chess), intrinsic limits / saturation (writing?), and knowledge limits (need new breakthroughs to exceed past performance). Better AI can help with computational limits, but not intrinsic limits. Work tests for orgs in the space: GiveWell lit review test Open Philanthropy work tests High-quality summaries of a conversation (AI is still failing our first-round work test at FRI about this) Abi: “I liked how [this] list broke down sectors as each having their separate indicators. Very societal frictions approach.” Measuring Adoption / Impact YouTube resulted in some homeowners not calling plumbers for small jobs. What are the AI equivalents, and how should they be measured? Compare metrics across AI-feasible vs. AI-infeasible sectors: employment (incl. hiring patterns + plans), productivity, profit, qualitative measures of AI usage, … Revenue of AI service providers Studies of AI impact across various fields What percent of sales of the U.S. pharmaceutical industry is generated by AI-discovered drugs and products derived from these? What percent of publications in the fields of Chemistry, Physics, Materials Science, and Medicine will be ‘AI-engaged’ as measured in this study or a replication study, in 2030? How many hours per week on average will K-12 students in G7 countries use AI-powered tutoring or teaching tools, as reported by their school systems or education ministries? A September 2024 study by the Federal Reserve Bank of St. Louis, based on the Real-Time Population Survey (N=3,216), estimated that 0.5%–3.5% of all U.S. work hours were assisted by generative AI. Measuring Diffusion Ajeya: I want to know about adoption within AI companies, so e.g. surveys about how much compute they’re running for inference, freedom of action related surveys like the ones suggested below, internal estimates of how much AIs speed them up, etc. (Also, I don’t think I literally care only about AI companies, will prob also care about USG.) Measuring Freedom of Action Surveys of workers - how often do you have to approve an AI decision, how much thought do you put into each approval (e.g. level of review / pass rate of AI pull requests) Monitoring progress on agent infrastructure → if AI agents are operating online, are there meaningful constraints on them? Does e.g. Cloudflare have any meaningful controls, or is it all a chaotic mess? Something military or military-adjacent - reports of smaller militaries using autonomous/semi-autonomous systems..? We can also make a big list of discrete flags and ask people about them, e.g. “Is your AI allowed to spend money on business expenses? Up to how much?” or “Is your AI allowed to search the internet freely in the course of completing a task?” or “Are AIs at your company allowed to talk to employees other than the human that started the task in the course of completing their task? What about external people?” White-hat reports of prompt injections paid out by bug bounties at the top (100) websites across domains over time (as a proxy for how many applications allow AI systems to take actions that can exfiltrate user data) Black-hat exploits of AIs (only possible if AIs are in a position to take important actions without adequate review?) Curve-Bending Mechanisms Note that the impact of AI can be difficult to predict. E.g. the prediction of AI flooding the zone with misinformation in the 2024 election didn’t pan out. Sycophancy → mental health impact, conversely, was an event that wasn’t widely predicted. Intelligence explosion (software and/or hardware) Measure both initial speedup, and whether r > 1 Surveys of AI companies on internal AI use (bunch of ideas for granular questions, e.g. subj sense of speedup, what tasks they’re used for now, what tasks aren’t they used for, hypothetical questions) Internal measures of the absolute rate of algorithmic progress at AI companies (e.g. how much compute does it take to get GPT-4-level perf), watch for that trend accelerating Source Ryan: Get people who have done the takeoff modeling to look at the data on algo progress (both historical from humans and data under automation regime). Unclear if this data will resolve disagreements, but seems like it can help. (Tom, Daniel, maybe some people from Epoch) Algorithmic breakthrough / new approach (e.g. neuralese, in-context learning, long-term memory, recurrence, brain emulation) (either known or unknown unknown) Study new model releases to look for signatures of known ideas, or look for an increase in papers on some approach Discontinuous jumps across a range of difficult benchmarks Could ask companies key questions like e.g. “are you still using English CoT?” and make them report Survey expectations at frontier labs (also covers intelligence explosion) Analysis attributing performance gains to different sources (e.g. model size scaleup, data quality improvement, RL, etc) Ryan: really big breakthroughs are like, AlexNet, GPT-1 / scale-up, scaling RL (o1, o3). Easy to notice something is turning into a big deal, hard to tell where on the Richter scale it will land. Could track the current 5 most promising potential trend-breakers. Ajeya: track breakdown of capabilities improvements over time: how much is from scaling pretraining, scaling RL posttraining, etc. Threshold effects / phase changes (e.g. the moment when people are no longer needed; maybe time horizon scaling suddenly becomes much easier) Phase changes in behavior of AI Village or other observations of AI capability / usage in real or realistic contexts Phase change in qualitative analysis of uplift trials Some resource (data, compute, electricity, intelligence headroom) is exhausted Note, may lead to workarounds rather than an end to progress. E.g. pretraining data gets harder to scale → scale RL instead. Hard to predict the effect. Low-hanging fruit is exhausted; training for long-horizon tasks is expensive Some important missing capability doesn’t yield to scaling + progress (e.g. sample-efficient learning, “judgement”, long-time-horizon skills) Economics of AI not working out → slowdown in investment AI winter – we run out of ideas for improving AI Josh: FRI expert panel. Could ask them “what’s the probability of an AI winter in the next 5 years”? External event (market downturn; war in Taiwan; public backlash / disaster → regulation; focused national effort / Manhattan Project) Track public sentiment and other political indicators Applications of models are a lagging indicator to model capabilities. First sign of a slowdown would be the time horizon graph slowing down. Other Nikola: if timelines are long enough to enable human emulations (Ems), that would make a difference, because we’d have been able to explore more implications. More generally, does other transformative tech come first. Possible techs: EMs, nanotech (unlikely), genetic engineering (seems like this could radically transform society in ~30 years if heavily invested in now, but in practice might not happen because of e.g. legal/regulatory blockers). [Child Page: Notes for Early Indicators] David Langer at Lionheart suggested that they might be in a position to help us measure some things. https://x.com/StringChaos/status/1928476388274716707 https://x.com/Simeon_Cps/status/1900218546904293863 Relevant indicators hinted at here: https://www.theinformation.com/articles/openai-says-business-will-burn-115-billion-2029 Example of finding useful data sets: https://mikelovesrobots.substack.com/p/wheres-the-shovelware-why-ai-coding Plan For this sort of project, serious work requires multiple rounds of deep thought. This can’t be done in a compressed period of time, so doing this collaboratively requires an extended multi-turn conversation. I have not had success in getting most people to engage with this in a group email or Slack. The only way I’ve found success is me doing 1:1 engagement with each participant, over email and occasional calls, with a lot of nagging. This is annoying but does seem to work. I’ve witnessed one counterexample (Helen’s CSET workshop) and heard about at least one other (something Sayash worked on); these were both collaborative projects centered around a multi-hour (or multi-day) synchronous group discussion. However, I’m not sure how many new ideas were generated at CSET, as opposed to just aggregating people’s existing thoughts. The CSET workshop did generate some new ideas but also there was a lot of aggregation. Maybe we should just embrace this: perhaps our niche is projects that require lots of multi-turn 1:1 interaction. Phase 2 of this specific project is more about aggregating ideas than deep analysis, so perhaps getting a group of people together on a Zoom to brainstorm together could work. Brainstorming and ideation was, I think, the focus of the two success stories I referred to. We could try to get a bunch of people on a call without being too fussy about exactly which people (though it would be really good to have someone from METR, someone from Epoch, and perhaps also the Forecasting Research Institute). This could be a meeting to put a cap on phase 1 (without necessarily completely locking it down, I’ll continue to get input from a few people I’m only connecting with now) and doing the initial brainstorming for phase 2. I can seed the conversation with a few ideas shared in advance. Note that our measurable indicators don’t need to directly answer a crux, we’re just looking for data that will be helpful in future discussions of those cruxes. → share thoughts with Taren, Ajeya, Helen, ? Ask Helen how far she’s gotten in turning the discussion into a paper and whether she wishes she could do more rounds with at least some folks. Ajeya: Might be productive to do this in two rounds. Do the fuzzy thing, then get someone who’s interested in doing the legwork to do the first round of coming up with concrete experiments – Ajeya might help with this – and take that back to the participants to rate them. Group lists the fuzzy question. Individual – Ajeya or Ryan Greenblatt – proposes a list of concrete experiments Go back to the group to rate the proposals. Realtime, concentrated bursts of time are so much more productive that it’s better to ask someone to come to an event that sounds cool & fun, spend the first hour reading the doc. Or can get people to pre-read more reliably if you get them to agree to come to an event, that’s what she did for the loss-of-control workshop. And keep the writeup short. Siméon (@Simeon_Cps) posted at 5:31 PM on Fri, Aug 15, 2025:I've entertained that theory for a few years and have been confused since then why people expected confidently so much GDP growth. Basically prices if goods should crash so fast that the question of "how do you count inflation" will become the first order parameter of whether and(https://x.com/Simeon_Cps/status/1956514153528742168?t=qJkEFlWoQAD6z6g4HMhNAw&s=03) Notes with Taren, 8/8/25 Post four brainstorming time slots, let people sign up; probably don’t want >4 people/session (though could do breakout rooms) Taren, and probably many people, will do better in a brainstorming session with people with different expertise Could let each group decide which two cruxes they’ll talk about; if we wind up with a gap, do something about it at the end. Or maybe assign cruxes to time slots. Doing an in-person session could be fun; might be more trouble than it’s worth, but might not be trouble at Constellation, ask them to help organize & recruit people, e.g. two 4-person groups Try to do one in the south bay? Could include Sam. Ask Ajeya how she’d like to participate Talk to Helen Toner, how to make sure what I’m doing is complementary Other people / groups to include DC – government does a lot of data gathering – propose content for legislation – Abi could help? Taren could do something in DC Oliver Stephenson (FAS) Elham, or the guy who worked for her? FAI IFP CHT (CBT?) FAS? Some Horizon fellows who are placed to draft a bill? Plan a third stage where we produce a piece of draft legislation? Discuss with Abi Discuss with Victoria next Friday Taren discuss with some people in DC Should discuss with Helen Toner Next steps: First step should be some Zooms, invite everyone. We won’t exhaust potential participants from Constellation. Taren to talk to Abi about whether to make this the theme for the dinner they’re organizing; if not, Taren will convene some other small meeting while she’s in DC. Pre-brainstorm: me, Taren, maybe Ajeya, maybe Abi? Chris Painter? Ryan Greenblatt? Josh Rosenberg? Someone with economics expertise (a grad student from David Otter’s lab)? More focus on identifying kinds of measurements (quantitative, qualitative, horizontal, vertical, etc.) to seed later conversations (+ as pre-read) Participants in Stage 2 Jaime Sevilla (Epoch) Content Notes See Untitled Review the Cruxes list in the early writeup Incorporate this idea from my discussion with Nick Allardice: Our economy and decision-making processes are so fragmented and messy. Trying to answer my crux questions at a societal level is unhelpfully generalizing. More tractable & beneficial is to pick some sectors, industries, types of problems, and come up with ways of measuring change in those, as leading indicators for what might happen in other sectors and industries. E.g. software engineering may be one of the more tractable problem spaces for AI; he’d be very interested in tracking diffusion here: hiring practices, how much autonomy is being granted, what level of productivity is it unlocking. If CS is a fast example, find a few slow examples, get super specific & granular about measuring diffusion. Get an idea of the uneven distribution. Carey: I don't know where this fits but I think the question of "what are the most common failure modes that prevent current models from excelling at practical tasks?" would be a relevant crux or root cause for your cruxes. Anna Makanju: It’s hard to break down someone’s usage of chatbots into productivity into other uses. In the last year it’s flipped from predominantly productivity usage into companionship. If you measure usage, need to try to disentangle in the nature of that usage – apply a classifier to their chat history, or focus on measuring enterprise accounts. Could work with universities and government agencies who will have enterprise accounts and might be more willing to share data for a study. Also see notes from my 8/11 conversation with Anna Why Focus on Early Indicators? If you want to make predictions about a future that is similar to the present, you might be able to simply extrapolate from past values of the variable you need to predict. For instance, Moore’s Law was an observation about trends in transistor counts, and for many decades it provided excellent forecasts of future transistor counts [FOOTNOTE: Though this may be a story about self-fulfilling prophecies as much as about the tendency of important variables to follow predictable trends.]. If you need to make predictions about a future that looks quite different from the present, you can’t get by with simple extrapolation. You need a model of how the future is going to unfold, and you need data to calibrate your model. For instance, if you want to predict the potential of fusion power, you can’t extrapolate the graph of historical electricity generation; for fusion, that graph is flatlined at 0. But if you understand the path that current fusion efforts are following, you can extrapolate metrics like “triple product” and “Q” [FOOTNOTE: I got help from Claude Opus 4 on this; the answer matches my vague recollection well enough that I’m not bothering to fact-check it: The most critical metrics for tracking progress toward practical fusion power are the fusion energy gain factor (Q), which measures the ratio of fusion power output to heating power input and must exceed 10-20 for commercial viability; the triple product (density × temperature × confinement time), which needs to reach approximately 10²¹ keV·s/m³ to achieve ignition conditions; and the reactor's availability factor or duty cycle, measuring the percentage of time the reactor can operate continuously, as commercial plants will need to run reliably for months at a time rather than just achieving brief fusion pulses.] to get an idea of how close we are to a functioning generator. Verify that an early application of the steam engine was to pump water out of coal mines. Make a reference to this being a sort of recursive self-improvement. Observe that if you had wished to measure the uptake of steam engines for pumping water out of coal mines, you could have looked at inputs to the process such as the amount of coal being consumed in steam engines outputs such as the amount of water being pumped or impact such as lowering the water level within the mines. When will “superhuman coders” and “superhuman AI researchers”, as defined in AI 2027, emerge? How is the task horizon identified in Measuring AI Ability to Complete Long Tasks progressing? What are we learning about time horizons for higher reliability levels (I presume reliability much higher than 80% will be necessary)? How fundamental is the gap between performance on benchmarks and real-world tasks? Is it growing or shrinking? [QUESTION: Does this cover “capability-reliability gap” described in AI as Normal Technology, or do we need to expand the description?] [this might better belong under the question regarding advances in domains other than coding and AI research.] Are any skills under coding or AI research emerging as long poles (more difficult to automate), and if so, are there feasible ways of compensating (e.g. by relying more on other skills)? What does the plot of AI research effort vs. superhuman performance look like? How does this vary according to the nature of the cognitive task? In particular, for tasks such as “research taste” that are critical to accelerating AI R&D? Basically redundant with previous cruxes, but perhaps worth listing as something that could be independently measured: across the broad range of squishy things that people do every day, how rapidly will the set of tasks for which AI provides significant uplift grow, and what are the contours of that set (what separates tasks experiencing uplift from tasks which are not)? Can AIs think their way around the compute bottleneck? If the R&D labor input races ahead of compute and data, how much progress in capabilities will that yield? To what extent does this depend on {quantity, speed, quality/intelligence} of the AI workers? Does this apply to all compute-intensive aspects of AI R&D? Is Ege’s forecast for NVIDIA revenue bearing out? Is his model for relating NVIDIA revenue to real-world impact of AI valid? Could rapid algorithmic improvements (driven by a software explosion) decouple impact from NVIDIA revenue? Could adoption lags result in revenue lagging capabilities? Possibly other questions as to whether various current trends continue – I’m not sure whether there are any other cruxes lurking here. For instance, is general progress in LLM capabilities speeding up or slowing down? Are breakthroughs emerging that accelerate the curve? Is RL for reasoning tasks hitting a ceiling? Are any of Thane’s bear-case predictions bearing out? Etc. As the automation frontier advances into increasingly long-horizon, messy, judgement-laden tasks, will the speed and cost advantages of AI (vs. humans) erode, to the point where they aren’t significantly faster or cheaper than humans for advanced tasks (and a few years of optimization doesn’t fix the problem)? Resources from the CSET conference Helen’s working doc My slides Ryan’s slides and notes It's important to detect when the most senior skills start to become automated. This could indicate a tipping point both for progress at the big Labs and or the ability for a breakout at a rogue actor who doesn't have access to senior talent. Perhaps we can look at the percentage of impactful ideas that come from unassisted AI. Perhaps we can look at the ratio of of major ideas, paradigm changing ideas to other inputs. Look for additional domains in which to measure sample efficient learning. In other domains, domains look at the ratio of spending on real world data collection versus in silico data generation. ARC Prize (@arcprize) posted at 10:21 AM on Tue, Sep 09, 2025:ARC Prize Foundation @ MITWe're hosting an evening with top researchers to explore measuring sample efficient in humans and machinesJoin us to hear from Francois Chollet along with a world class panel: Josh Tenenbaum, Samuel Gershman, Laura Schulz, Jacob Andreas https://t.co/dq7NJyXkNk(https://x.com/arcprize/status/1965465501079142814?t=L7y6E9f9cwxgjwWD1FMZJA&s=03) https://epochai.substack.com/p/after-the-chatgpt-moment-measuring Check in with Divya / CIP to see whether their global pulse surveys have questions relevant to the cruxes. Perhaps we can draw from this / make suggestions for it. They’re running “global pulse” surveys, to understand what people want from the future but also to understand how much AI has diffused into people’s lives. Every two months, started in March. Questions around trust, diffusion, how much are you relying on AI for medical or emotional advice, are you using it in the workplace, etc. https://globaldialogues.ai/cadence/march-2025, “In three years, what questions will we wish we had been tracking?” Maybe could be interesting to co-author something at some point. Cheryl Wu (@cherylwoooo) posted at 6:25 PM on Sun, Jun 01, 2025:Are we at the cusp of recursive self-improvement to ASI? This tends to be the core force behind short timelines such as AI-2027. We set up an economic model of AI research to understand whether this story is plausible. (1/6)(https://x.com/cherylwoooo/status/1929348520370417704?t=f9E9Yty2m27EQ-Z_NbbJ3Q&s=03) From Does AI Progress Have a Speed Limit?, a measurement: Ajeya: I'm kind of interested in getting a sneak peek at the future by creating an agent that can do some task, but too slowly and expensively to be commercially viable. I'm curious if your view would change if a small engineering team could create an agent with the reliability needed for something like shopping or planning a wedding, but it's not commercially viable because it's expensive and takes too long on individual actions, needing to triple-check everything. Arvind: That would be super convincing. I don't think cost barriers will remain significant for long. Another: Ajeya: Here's one proposal for a concrete measurement — we probably wouldn't actually get this, but let's say we magically had deep transparency into AI companies and how they're using their systems internally. We're observing their internal uplift RCTs on productivity improvements for research engineers, sales reps, everyone. We're seeing logs and surveys about how AI systems are being used. And we start seeing AI systems rapidly being given deference in really broad domains, reaching team lead level, handling procurement decisions, moving around significant money. If we had that crystal ball into the AI companies and saw this level of adoption, would that change your view on how suddenly the impacts might hit the rest of the world? Another: Ajeya: …do you have particular experiments that would be informative about whether transfer can go pretty far, or whether you can avoid extensive real-world learning? Arvind: The most convincing set of experiments would involve developing any real-world capability purely (or mostly) in a lab — whether self-driving or wedding planning or drafting an effective legal complaint by talking to the client. From Deric Cheng (Convergence Analysis / Windfall Trust) Early indicators: he’s friends with the Metaculus folks. They’re working on indicators for AI diffusion and disempowerment. Don’t share: Metaculus Diffusion Index – they’ll publish in a few weeks. https://metr.substack.com/p/2025-07-14-how-does-time-horizon-vary-across-domains When monitoring progress in capabilities, need to watch for the possibility that capabilities are advancing on some fronts while remaining stalled on some critical attribute such as reliability, hallucinations, or adversarial robustness Nick Allardice has a prior that labor market disruption is not going to be meaningfully different from other times in history… but he’s highly uncertain. Evidence that might push him to believe that meaningfully different levels and pace of labor market disruption would [?]. The burden of proof is on this time being meaningfully different from the past. If AI gets good at something, we’ll focus on something else. He hasn’t seen enough evidence to shift his priors. Leading indicators currently reinforce his prior: unemployment is low. It’s harder to get a job as a junior developer, but not impossible, and mostly seems to be due to other factors. Even if capabilities advance, diffusion challenges will leave room for human workers. Our institutions aren’t going to turn everything over to AI. Offcuts qualitative rubrics [MPH is a good indicator because every M takes about the same number of H; cities passed per hour would break down in the Great Plains; MPH breaks down when you enter a city center] If we only measure high-level, downstream attributes such as practical utility, we won’t have any way of anticipating these twists and turns. As I noted in the introduction to the cruxes writeup, by the time Facebook started to show up as a major contributor to overall Internet usage, it was already well along its journey to global dominance. [Child Page: Seeing Past AGI; AI 2027 vs. AINT] Summary: Engage with Ryan Greenblatt and others on what I’m now calling “Seeing Past AGI”. Ryan keeps pointing out that there are important disagreements which won’t manifest until after AGI is reached, and which may be hard to shed light on until that point (which may be too late to be of much use). Working with Ryan and other usual suspects, I’d be interested in digging into this, try to clearly characterize the dramatic things Ryan expects to see happen post-AGI, and identify precursors which Ryan would agree ought to be visible pre-AGI. This might turn out to fit neatly into the existing AI Watersheds framework, or might turn into a bit of a separate sub-project. As we close in on the relevant milestones, we won’t resolve the first two bullets above [I think this meant the first two cruxes]. The difficult questions are not when X gets automated, it’s what happens afterwards. Can’t imagine any measurement that would disambiguate AINT for him. A lot of these metrics do shed light on whether / when we’ll get AGI, or automation of AI R&D. I have a hard time imagining updating toward anything like “full cheap automation of cognitive labor would increase GDP growth by <10%” I have a hard time imagining updating against large impacts of ASI in advance I’d like to push on this and try to come up with ways to forecast past the AGI event horizon – I have a conviction that we can do better than just throwing up our collective hands. My plan for a next step is to engage with, perhaps, Ryan and Sayash to really dig into their models of what happens post-AGI. [Abi: Let’s chat about different definitions of AGI that Sayash and Ryan have. I wonder whether getting very specific on this question will point to two very different visions here, which are leading to some of the divergence. + how we handle this in our approach.] (related: per https://blog.ai-futures.org/p/ai-as-profoundly-abnormal-technology, offer to meditate a conversation between Sayash+Arvind and the AI 2027 crew) If/when we pursue this: Think about Ryan’s comment about takeoff in the phase 2 doc. Come up with a plan for drilling in on takeoff models and early indicators. Work with Sayash and Ryan to drill in on disagreements post-AGI and then look for early indicators Abi: During our call, let’s chat about different definitions of AGI that Sayash and Ryan have. I wonder whether getting very specific on this question will point to two very different visions here, which are leading to some of the divergence. + how we handle this in our approach https://blog.ai-futures.org/p/ai-as-profoundly-abnormal-technology https://x.com/sayashk/status/1964016339690909847 [Ryan, under “measuring freedom of action”] It seems really hard to use indicators to distinguish between my perspective and one that predicts way less freedom of action at the point of ~full automation of AI R&D. Maybe the crux is mostly capabilities, but then it comes back to crux 1. Ryan: as we close in on the relevant milestones, we won’t resolve the first two bullets above. The difficult questions are not when X gets automated, it’s what happens afterwards. Can’t imagine any measurement that would disambiguate AINT for him. A lot of these metrics do shed light on whether / when we’ll get AGI, or automation of AI R&D. Delaying timelines by decades is a big deal, but doesn’t have a decisive effect on what I ultimately expect to happen. E.g., it’s like a factor of 3 on various things, not a factor of 10-100. More random takes from Ryan: I can imagine updating towards much longer timelines (though this is limited by some exogenous rate of large breakthroughs) I can imagine updating away from software-only singularity based on detailed empirical evidence about returns to compute vs labor within AI companies (especially if this evidence is coming in as we’re automating) Though idk how big this update would be. I can imagine updating towards moving through the human range slower than I currently expect via a variety of mechanisms. I have a hard time imagining updating toward anything like “full cheap automation of cognitive labor would increase GDP growth by <10%” I have a hard time imagining updating against large impacts of ASI in advance After this happens, I’d update, but this is too late. AI Scenarios Network – AI Watersheds Brainstorm Prioritize putting together a list of measurements, so I can ask for additional suggestions + then do a round of voting Turn phase 2 notes into a rough draft Early writeup Group brainstorm Steve’s Notes from the Phase 2 Brainstorming Doc Slide deck Intro Material for Phase 2 The importance of measurements that can be collected over a long period of time (won’t saturate), and ideally can be measured historically as well. [more ideas from the panel will belong here… using fine-grained data sets, collecting data from inside labs, using LLMs to analyze qualitative data, setting up controlled access to sensitive data sets, etc. Maybe these detailed ideas of “how to measure things” would belong in a separate section later on, and up here we’d just talk about basic philosophical ideas like “focus on real-world impact”.] https://ai-frontiers.org/articles/the-hidden-ai-frontier talks about how important developments may be hidden inside the frontier labs (and the dangers that poses) General Material for Phase 2 File for AI Watersheds, eg "Concretely, one reviewer proposed tracking deployments of AI agents that (i) are general-purpose systems, (ii) operate with minimal supervision, and (iii) handle tasks with a high cost of errors.” gavin leech (@g_leech_) posted at 4:00 AM on Thu, Nov 13, 2025:Glad somebody did this (expert interviews on why LLMs are not currently AGI, and why they could be)feat: @random_walker, @DKokotajlo, @ben_j_todd, @daniel_d_kang, @rohinmshah https://t.co/MuHjAoXvAw(https://x.com/g_leech_/status/1988939922842218558?t=OSiH9KzQDuAzCWo776O6Vg&s=03) @AI Security Institute: 📈 Today, we’re releasing our first Frontier AI Trends Report: evaluation results on 30+ frontier models from the past two years, showing rapid progress in chemistry and biology, cyber capabilities, autonomy, and more.▶️Read now: https://t.co/afJoJy0pYl pic.twitter.com/uIBmxQMNjJhttps://x.com/i/status/2001579052830953668 https://x.com/sayashk/status/1963343022252315112 https://epochai.substack.com/p/the-changing-drivers-of-llm-adoption Follow up on “AI Watersheds Phase 1 writeup” with Jonas Sandbrink; lots of good ideas in his email, and we should discuss further. https://x.com/EpochAIResearch/status/1996248575400132794 Herbie Bradley (@herbiebradley) posted at 5:13 AM on Thu, Nov 20, 2025:Looks like a very promising benchmark(https://x.com/herbiebradley/status/1991495140633141550?t=B2ssdv1dhSdNGdMxxOJ6TA&s=03) Might incorporate: https://arxiv.org/pdf/2510.07575v1 [Abi] Geopolitics + AGI team at RAND takeaways: This team also sees their goal as getting decisonmakers (not just gov) to think more about TAI. ON BOTTLENECKS TO ECON DATA: Their econ team said that lack of granular data constrains econ research. Example: o-net's list of tasks is great but needs more granularity and to be updated more. O-net surveys are only send sporadically to ~5 people per firm. Example: The Census collects longitudinal employer/employee data ("LAHD") but only 25 states opt in. Also, researchers need special status to use it. RAND has access but notes that it's not even that good because occupation data isn't linked to worker-firm datasets. This might be relevant to Watersheds! TO DOs: I will add a chat with RAND's AGI NatSec team to the Director of Event's onboarding doc. If desired in January, I can set up a chat with someone from RAND Econ team to talk to Taren or Steve about how to unlock from task-level or occupation-level data. https://epochai.substack.com/p/the-software-intelligence-explosion Are these trends bearing out? https://x.com/Hangsiin/status/1950645770346283083?t=2LsNFOIyCZy2Ar22eUR-iA&s=03 Are these cognitive limitations easing? https://leap.forecastingresearch.org/reports/wave2 Notes from Jonas: He sees the key disagreements as feasibility of ASI, and speed of diffusion (and will AGI increase or decrease barriers to adoption). [feasibility of ASI → potential bend in the curve, is this on our list?] If you extrapolate the METR curve, you’re probably not looking at AGI in 2028. The only way to get there soon is through some sort of speedup. So if you want to evaluate whether AGI is coming within a few years, you should be looking for signs of speedup. If you expect AGI in more like 2035, you should be looking at broader progress. How will we measure AI capabilities for multi day tasks? From https://digitaleconomy.stanford.edu/news/ai-and-labor-markets-what-we-know-and-dont-know/: One way to bolster evidence is to collect better data on when individual firms adopt AI (see more in point 4) to track employment changes before and after at the firm level, hopefully improving upon the measures in Humlum and Vestergaard (2025), Hosseini and Lichtinger (2025), and other work. Even better would be to find some kind of experiment in firm-level AI-adoption. An example would be an A/B test at an AI company that randomly offered discounts on subscriptions to different firms. Ideally the experiment would have been run starting in the early days of AI and run for months, if not years. It would be great to get actual large-scale data from AI labs on usage by occupation, perhaps via survey rather than relying on predictions based on conversations. More research should be done on other labor markets. Three promising avenues are to use Revelio or ADP in other countries, if feasible; use other private payroll data from other countries; or use government administrative data to track employment changes. Some infrastructure likely needs to be built out to measure AI exposure for local occupations. A particular area of focus should be countries with high levels of employment in exposed jobs such as call center operations. Further modeling can also help with predicting how impacts may vary across different institutional contexts. Ideally we would have some sort of continuous index of AI adoption, with differences in “how much” firms or workers have adopted AI. One option is to measure token counts, as suggested by Seed AI. Business spend data seems promising as well. Another option is the number of unique users or the number of conversations. We should encourage AI companies to share data on this to the extent feasible. Business surveys should also explore alternative questions and test how sensitive reported adoption rates are to the specific wording. Not related to cybersecurity, but we did a deep dive on agent benchmarks. Many of them are broken and measure AI agent performance poorly: Twitter/X: https://x.com/daniel_d_kang/status/1942641179461648629LinkedIn: https://www.linkedin.com/posts/daniel-kang-1223b343_ai-agent-benchmarks-are-broken-activity-7348406954253312000-B_Yq/Substack: https://ddkang.substack.com/p/ai-agent-benchmarks-are-broken Remotelabor.ai to track the new Remote Labor Index, measuring what percentage of remote work AI can automate. Currently the top score is 2.5%, so ‘not much,’ but that’s very different from 0%. Diffusion: https://www.convergenceanalysis.org/fellowships/spar-economics/decoding-ai-diffusion-mapping-the-path-of-transformative-ai-across-industries Ethan Mollick (@emollick) posted at 5:09 PM on Thu, Sep 25, 2025:After reading it, this does seem like a big dealIndustry experts outlined important, real-world, hard tasks for AI to do. Other experts were asked to do the tasks themselves & yet others graded human & AI outputModels approached parity with humans & AI is getting better fast. https://t.co/z666YcNyH6(https://x.com/emollick/status/1971366497244348625?t=suZJLFoe4S9wZoSROiNeKw&s=03) OpenAI (@OpenAI) posted at 9:24 AM on Thu, Sep 25, 2025:Today we’re introducing GDPval, a new evaluation that measures AI on real-world, economically valuable tasks.Evals ground progress in evidence instead of speculation and help track how AI improves at the kind of work that matters most.https://t.co/uKPPDldVNS(https://x.com/OpenAI/status/1971249374077518226?t=aXFDp9V1lvUVuMBJOotUFA&s=03) Lawrence H. Summers (@LHSummers) posted at 9:36 AM on Thu, Sep 25, 2025:A research team at @OpenAI, where I am proud to be a board member, released an important new paper today. This paper looks at what might be thought of as task specific Turing Tests and shows that AI systems, even with limited guidance, perform many tasks -- such as planning(https://x.com/LHSummers/status/1971252567981146347?t=dnQgGIFT7yFz-Ex3KHVhpA&s=03) Sayash Kapoor (@sayashk) posted at 1:54 PM on Wed, Oct 15, 2025: 📣New paper: Rigorous AI agent evaluation is much harder than it seems. For the last year, we have been working on infrastructure for fair agent evaluations on challenging benchmarks. Today, we release a paper that condenses our insights from 20,000+ agent rollouts on 9 https://t.co/TvSxUsptdW (https://x.com/sayashk/status/1978565190057869344?t=sQeizZZnd9uOH-Chjv8e5A&s=03) Dan Hendrycks (@DanHendrycks) posted at 7:20 AM on Thu, Oct 16, 2025:Our definition of AGI is an AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult.To measure this, we assess the multiple dimensions of intelligence derived from the most empirically validated model of human intelligence (CHC theory). https://t.co/e1wEkzmHwb(https://x.com/DanHendrycks/status/1978828383581561009?t=b6nlhd1Uh6Xj57adoDnzEQ&s=03) Ethan Mollick (@emollick) posted at 10:24 AM on Thu, Oct 16, 2025:A lot I like & some I don’t in this paper:Like: Clear definition of AGI, diverse authors, shows jaggedness, tracking metrics over time (huge leap from GPT-4 to GPT-5)Dislike: AGI defined as replicating a model of human cognition, benchmarks are scattershot, narrow view of AI https://t.co/T3XOu2PVl8(https://x.com/emollick/status/1978874737892667718?t=hWtw4waaXLy-Xa4djFU-iQ&s=03) Sayash Kapoor (@sayashk) posted at 10:12 AM on Fri, Sep 12, 2025:Agent benchmarks lose *most* of their resolution because we throw out the logs and only look at accuracy.I’m very excited that HAL is incorporating @TransluceAI’s Docent to analyze agent logs in depth.Peter’s thread is a simple example of the type of analysis this enables,(https://x.com/sayashk/status/1966550402129592738?t=SX4UR2z0FabBX_mgLc98dw&s=03) The Point Magazine (@the_point_mag) posted at 6:05 AM on Thu, Oct 16, 2025:New online, @saffronhuang on what it means to measure intelligence—in large language models and in us:https://t.co/AKZdHiyzGE(https://x.com/the_point_mag/status/1978809403609382977?t=pzJVLmXj6ZGb3k88X648qQ&s=03) At the CAIS event on Oct. 2, someone (Dan?) mentioned that they’d be posting an AI Automation Index in a few weeks Alex Tamkin (Anthropic Economic Index coauthor): data sources: model usage data (can't be longitudinal because don't keep logs, also confounded by changing models), government (states might be a good source if can't get federal help), downstream apps, Stripe. He'd love to see interviews, eg of hiring managers. Bharat: someone in his group wanted to know the capital vs labor contribution to AI R&D, would be helpful in calibrating the model, that's the missing variable for his model. Miles Brundage: How confident is he in short timelines? Pretty confident. He’s typical of people who have spent multiple years at a frontier AI company and lived through / closely watched / participated in multiple scaleups, going from “signs of life” to maturity stage for image generation, codegen, video gen, writing, math. We’re still early on RL, and even pretraining – for instance, we’ve barely scratched the surface on video data (YouTube). On fuzzy vs. tidy problems: he views these as differences of degree vs. kind. E.g. there’s a lot of positive spillover from math RL to code, and code RL to writing or policy research. There’s very little data relating to papers he writes in OpenAI’s RL, but chain of thought induces useful skills (such as breaking problems down into parts, checking your work)… that makes it useful for working on his papers? It just takes intellectual labor to turn a non-verifiable task into something you can test and verify. [Josh 6/9/25] My colleague Alexa (cc'd) put together some initial ideas for forecasting questions that could help to further specify and concretize some of the cruxes described in your post. She gives more context on the approach in the summary of her document. Would you be interested in incorporating a revised version of any of these questions into your work, or possibly trying to get forecasts on them as part of your research? We'd also be happy to help with collecting forecasts if you'd find that valuable. Dean W. Ball (@deanwball) posted at 9:09 AM on Sun, Sep 14, 2025:I think Demis is fundamentally correct here. The current systems are extremely impressive, and will get much more so soon, but it’s clear there are fundamental breakthroughs still needed.As I have written before, I expect us to get “superintelligence” (AI systems that can, say,(https://x.com/deanwball/status/1967259417029837122?t=slJA0l1BoEy3WQdOJzlUGQ&s=03) Dean W. Ball (@deanwball) posted at 5:09 AM on Tue, Sep 16, 2025:If this mirrors anything like the experience of other frontier lab employees (and anecdotally it does), it would suggest that Dario’s much-mocked prediction about “AI writing 90% of the code” was indeed correct, at least for those among whom AI diffusion is happening quickest.(https://x.com/deanwball/status/1967923900685386222?t=FztCAYh5PbN51JLbzAcGUA&s=03) Steven Adler @ Progress Conference 2025 Talked about looking at UpWork task mix, prices, etc. as a signal Talked about building evals around more open ended real work tasks. 1a3orn (@1a3orn) posted at 4:20 PM on Sat, Oct 25, 2025:data from OpenAI / Anthropic that I wish I had, but do not:1. What percent of Transformer improvements in OAI / Anthropic are original to the company, and what percent come from outside?2. What "Constitutional principles" does Anthropic currently use for alignment?(https://x.com/1a3orn/status/1982225866470899728?t=BxaB8hw-2u-HkAGBlH93bg&s=03) https://thezvi.substack.com/p/asking-some-of-the-right-questions Could talk to Gabe Weil, who mentioned that Basil Halperin argues that AGI should raise interest rates. https://newsletter.forethought.org/p/how-quick-and-big-would-a-software https://tecunningham.github.io/posts/2025-09-19-transformative-AI-notes.html Incorporate ideas from my chat with Jaime Will investor confidence continue to support scaling of compute / training budgets? Would love to have more visibility into the revenue chain, how solid the demand is, and where the room for short- and long-term growth comes from. We talked about my question about how revenues flow through the AI value chain and how much of this is speculative vs. committed users who are experiencing values. Jaime noted that Anthropic is very dependent on coding tools. Investors are ready to fund roughly 3 years of burn, so perhaps 10x ARR (at current growth rates). To keep up this rate of growth, it’s necessary to keep expanding into new markets. What’s the penetration rate of coding tools? Jaime is surprised how low… anecdotally, 6 months ago talking to random developers in Spain and Mexico, no one is really using these tools professionally. Areas he expects to see impact soon: accounting, customer service, legal work, finance, assistants / operations, market research analysts. Seems like there’s plenty of room here for a couple more years of revenue growth at the current rate. What would be useful for looking more than a couple of years ahead? He likes to take an outside view, look at what other markets are exposed. A colleague went through the OINT (?) database and made a list of exposed occupations. They could do this at larger scale. What future capabilities will be needed? Can guess, e.g. much better computer use. His big disagreement with AI 2027 is around returns to intelligence and returns to parallelization of research. He doesn’t foresee nearly the same degree of benefit from running lots of small experiments in parallel. One thing that has shaken his beliefs on returns to intelligence is the insane amount that companies are willing to pay their top researchers. Difficult to interpret exactly what that means, but might suggest returns to intelligence. https://blog.cip.org/p/notes-on-building-collective-intelligence Pass @ kitchen sink: https://www.notion.so/Todo-5cbf5bb74635457381c2f814628c73f9 Notes from Jason Clinton’s talk at The Curve 2025 (John Hart may have more); could incorporate these into metrics of internal usage at the labs and how this differs from other orgs: Anthropic has automated level 1 SOC analysts: reviewing alerts to decide if they require action. 2-3 months from automating tier 2: deciding which alerts are too noisy. Blocked on visual reasoning??? Human code review is 25% effective at catching security bugs. AI could be better. AI reviewer writes a repo before reporting the issue to a person. An AI is greenlighting 60% of design docs as being low risk with no need for human security review. John’s notes 90% of code at Anthropic is written by Claude; they have eliminated all junior dev roles from their open jobs list "vibe hacking" - 3 weeks ago a "low-skill russian hacker" used Claude to hack people - https://www.bbc.com/news/articles/crr24eqnnq9o DARPA just concluded an "AI Cyber Challenge" - https://aicyberchallenge.com They have a specialized Claude agent that focuses on flaky tests. If it gets stuck / can't un-flake a test, it will "reach out" to a "Claude SRE" agent to see if it's infra-related. Likewise they have a specialized Claude agent that just does security review of design proposals; this has offloaded some routine work from their principals. It compares design docs against all known MITRE attacks. Claude does all Tier 1 SOC analyst work; only when it raises to Tier 2 level (arbitrating noisy alerts, for example) does a human get in the loop "Literally everyone is working on memory". Multiple startups will be offering "virtual employees" (presumably w/ long-term context memory) starting in April or May of next year. Responsible disclosure timelines are woefully out-of-date in the AI era. https://ai-frontiers.org/articles/the-hidden-ai-frontier talks about how important developments may be hidden inside the frontier labs (and the dangers that poses) Per discussion with Nick Allardice, impact will play out very differently (and more slowly) in the global south. Research priorities I’m excited about this. I think our “neutral/switz” angle can help with this. On the collective action, could potentially frame it in labs’ interest as a way to get better forecasts, maybe pair with some of the platforms where leading labs already are i.e. Coalition for Secure AI, maybe GPAI. Propose some specific initiatives. Emphasize projects that require collective action, such as: A large effort to collect a valuable data set which would be useful for multiple research projects Collecting data that requires cooperation from AI labs or other private sources, because the data is sensitive and/or requires effort for the private actor to supply. Collective action may be needed to pressure the labs into cooperating, and/or to create a high-trust context in which appropriate safeguards can be provided (controlled access to data). === Steve’s Notes from the Phase 2 Brainstorming Doc (https://www.notion.so/2641a29f1789802f9e0cfcc9eccabb24) === Brainstorming doc Why Do Measurements? Thoughts on the purpose of forecasting AI’s schedule General trend visibility for several years (important for lots of policy + other questions) At least several years notice of transformational change Estimate likely steepness & extent of transformational change Ask panel for feedback on this Josh: interested in the connection between current policy questions and our questions here. Would be great to get more specific / explicit about the connections with policy questions. Josh: interested in the connection between current policy questions and our questions here. Would be great to get more specific / explicit about the connections with policy questions. General thoughts on what to measure Practical utility is a function of both AI model capability and complementary app innovation (coding models vs. Cursor) Approaches to measurement Josh (+’d by Helen, Nikola, Sayash): Noting cross-cutting theme: Generally seems like more high-quality surveys of companies would be valuable. A lot of ideas involve sensitive data, so would require controlled access (and/or governmental involvement?) Using AIs to qualitatively grade lots of screen recordings and other messy high-volume data Find ways of calibrating / correlating benchmark scores with measurements we care about more directly? (METR time horizons being an example in this direction) Generally use lots of different measurements and try to map them to common units for cross-checking Surveys / prediction markets; focus real-world measurements (e.g. uplift studies) on areas of disagreement Or get both sides of a dispute to agree on a cheaper proxy measurement Case studies How firms in various sectors are using AI; scale of tasks assigned to AI / nature + granularity of interaction with humans; degree of trust / freedom of action given to AI Deep study of revenue / spending / profit across the AI value chain (and by domain) Also: number of “complementary innovation” (Sayash’s term) AI-powered apps, adoption rate, # of customers, “Surveys of customers to see how they are improving their workflows”, etc. Helen: something about revenue from AI more broadly vs. revenue of frontier model providers. What can we say about open models eating into OpenAI / Google / Anthropic revenue? Broad metrics (a la Epoch), e.g. electricity used by AI, global private investment Keep pushing benchmarks toward greater realism of both input (task specification + context) and output (not just “passes tests”, but “PR is mergeable” / other measures of quality) Nikola: need more realistic benchmarks. Source PRs (ideally from inside AI labs) and test AIs on them. Analysis of usage at the model level (Anthropic economic AI index) Logs of agent usage i.e. usage of model context protocol servers, pulls [Sayash] Analysis of AI logs from them being asked to solve various tasks (we’re doing this with 10B tokens of HAL agent log data, but could imagine increasing many fold) Social media is subject to transparency measures, AI companies currently are not. It’d be great to have controlled researcher access to e.g. ChatGPT transcripts. Lots of general data sets for public analysis AIs really like to email Eliezer, that’s an indicator of various types of strange AI behavior. Ajeya: there was an NYTimes article about a journalist getting similar emails. Could there be a “weird AI shit” incident tracker that would categorize reports + report trends? Josh: MIT AI incident repository? Populated from public reports – links to existing reports. Ajeya: hard to make sense of it. Needs to be a larger team sifting the signal from the noise + writing blog posts. Doesn’t track veracity. AI Risk repository / incidents database is relevant here: https://airisk.mit.edu/, also https://incidentdatabase.ai/ Ajeya comment: Too hard to make sense of current databases. Need someone to do better synthesis. Make an open source AI company where the company itself is open source. Have it try to use AI as much as possible, make all its logs about everything (Slack etc) public, study that. Sayash: there’s been a lot of work on the science of AI evaluations. We don’t know the shape of the DAG, how AI inputs translate into outputs. We could do more to distinguish between different DAGs – which is a core source of disagreement, for instance Sayash has a very different DAG in mind. Inputs might include things like data, compute; output measured in the real world. Econometrics. Josh: Meta measurements: Big increase in expected OpenAI revenue, according to a prediction market, based on a reported algorithmic breakthrough Pushback from Ryan+Ajeya: limitations of prediction markets mean it’ll be tough for them to pick this up reliably Surveys at frontier AI companies Surveys of the broader workforce Data center construction spend → currently dependent on media reporting, is there some way to get better/more consistent data on this? Trying to unpack inputs/outputs per Sayash’s point about DAGs/understanding how improvements in one element flow through to improvements in other elements Lots of (different types of) surveys Helen: important to track military applications (broadly – decision support, targeting, etc.). Important to track freedom of action here. Measuring Utility Metrics about fluid intelligence, messy tasks Benchmarks for domains vs. skills Domains: cyber/law/professional labor markets Skills: Context awareness, reliability, long-context reasoning, sample efficiency Need deep info for each sector before assessing results. I.e. persuasion surveys overestimate results because opinion change dissipates 6 weeks later. I like Steve’s note that AI can continually be used cheaply. Uplift RCTs in real-world settings, measuring real-world outcomes (ideally: profit!) e.g. if you get an actual large company to actually do randomized staged rollout of enterprise LLMs that’s great Ajeya: three-armed uplift trials, AI / human / cyborg. Josh: Persuasion: Cost for AI to swing a vote, relative to other leading interventions (Josh: lots of polysci studies here) AI-bio: % of amateurs that can synthesize flu Look for leapfrog companies that get on board with a new technology ahead of the incumbents, that could highlight utility ahead of adoption. e.g. fintech cos using AI in ways that let them overtake traditional banks, or healthcare startups using AI in ways that let you bypass doctors’ offices Power user case studies – screen recordings + deep interviews of people who (think they) are getting enormous uplift Organization version of this: find those YC startups that supposedly do 95% of everything with AI, study them (“power organizational users”) Field trials (a la AI Village) and realistic benchmarks Do a time horizon analysis (a la METR), but on real-world usage? Also, qualitative analysis of the size + difficulty + nature of tasks successfully delegated to AI (and dig into ranges of “successfully”) Building on Josh’s work test idea: could a METR-like org collect work test setups from a range of places (under NDA so as not to break the work tests) and centralize the work of eliciting good performance, figuring out measurement, reporting results, etc? Ryan: I think uplift will be messy (because of humans being messy and possibility for phase changes in how AI should be used productively), so I’m more into end-to-end automation of actual tasks. Also, we ultimately care most about full automation regimes and predicting when this will happen. Uplift studies correspondingly seem somewhat worse than looking at full automation of tasks of varying difficulty/size (and benchmark transfer to actual things people are doing in their job so we don’t need to constantly run these tests). Maybe you can get AI companies to run semi-informal experiments with uplift internally and publish results: many companies have effectively committed to doing this eventually due to their safety policies. I think Anthropic might open to this, but there are various sources of trickyness here. Early reports of AI doing remarkable things (e.g. major scientific discovery / insight; solving a Millennium Prize Problem in mathematics) pass@any for existing benchmarks can predict pass@1 in the future Helen: where is the ceiling on various capabilities (headroom)? This seems under-investigated. Sayash: are most tasks like chess, or like writing? (Ryan: unclear whether writing is the right contrast here.) Are there less controversial examples than writing? Taking out the trash: saturates. Helen: the writing example highlights we’re way under-theorized on this. Quality of parenting is another example. “Persuasion” is poorly specified. Abi: a scientific breakthrough requires an insight that goes against dogma. Could be considered an aspect of very high-quality writing. Ryan: I care most about questions like “will energy production 100x within 5 years of full AI R&D automation if people want this to happen”. Seems like this has many possible routes in terms of capabilities head room. Sayash: three broad categories: computational limits (chess), intrinsic limits / saturation (writing?), and knowledge limits (need new breakthroughs to exceed past performance). Better AI can help with computational limits, but not intrinsic limits. Work tests for orgs in the space: GiveWell lit review test Open Philanthropy work tests High-quality summaries of a conversation (AI is still failing our first-round work test at FRI about this) Abi: “I liked how [this] list broke down sectors as each having their separate indicators. Very societal frictions approach.” Measuring Adoption / Impact YouTube resulted in some homeowners not calling plumbers for small jobs. What are the AI equivalents, and how should they be measured? Compare metrics across AI-feasible vs. AI-infeasible sectors: employment (incl. hiring patterns + plans), productivity, profit, qualitative measures of AI usage, … Revenue of AI service providers Studies of AI impact across various fields What percent of sales of the U.S. pharmaceutical industry is generated by AI-discovered drugs and products derived from these? What percent of publications in the fields of Chemistry, Physics, Materials Science, and Medicine will be ‘AI-engaged’ as measured in this study or a replication study, in 2030? How many hours per week on average will K-12 students in G7 countries use AI-powered tutoring or teaching tools, as reported by their school systems or education ministries? A September 2024 study by the Federal Reserve Bank of St. Louis, based on the Real-Time Population Survey (N=3,216), estimated that 0.5%–3.5% of all U.S. work hours were assisted by generative AI. Measuring Diffusion Ajeya: I want to know about adoption within AI companies, so e.g. surveys about how much compute they’re running for inference, freedom of action related surveys like the ones suggested below, internal estimates of how much AIs speed them up, etc. (Also, I don’t think I literally care only about AI companies, will prob also care about USG.) Measuring Freedom of Action Surveys of workers - how often do you have to approve an AI decision, how much thought do you put into each approval (e.g. level of review / pass rate of AI pull requests) Monitoring progress on agent infrastructure → if AI agents are operating online, are there meaningful constraints on them? Does e.g. Cloudflare have any meaningful controls, or is it all a chaotic mess? Something military or military-adjacent - reports of smaller militaries using autonomous/semi-autonomous systems..? We can also make a big list of discrete flags and ask people about them, e.g. “Is your AI allowed to spend money on business expenses? Up to how much?” or “Is your AI allowed to search the internet freely in the course of completing a task?” or “Are AIs at your company allowed to talk to employees other than the human that started the task in the course of completing their task? What about external people?” White-hat reports of prompt injections paid out by bug bounties at the top (100) websites across domains over time (as a proxy for how many applications allow AI systems to take actions that can exfiltrate user data) Black-hat exploits of AIs (only possible if AIs are in a position to take important actions without adequate review?) Curve-Bending Mechanisms Note that the impact of AI can be difficult to predict. E.g. the prediction of AI flooding the zone with misinformation in the 2024 election didn’t pan out. Sycophancy → mental health impact, conversely, was an event that wasn’t widely predicted. Intelligence explosion (software and/or hardware) Measure both initial speedup, and whether r > 1 Surveys of AI companies on internal AI use (bunch of ideas for granular questions, e.g. subj sense of speedup, what tasks they’re used for now, what tasks aren’t they used for, hypothetical questions) Internal measures of the absolute rate of algorithmic progress at AI companies (e.g. how much compute does it take to get GPT-4-level perf), watch for that trend accelerating Source Ryan: Get people who have done the takeoff modeling to look at the data on algo progress (both historical from humans and data under automation regime). Unclear if this data will resolve disagreements, but seems like it can help. (Tom, Daniel, maybe some people from Epoch) Algorithmic breakthrough / new approach (e.g. neuralese, in-context learning, long-term memory, recurrence, brain emulation) (either known or unknown unknown) Study new model releases to look for signatures of known ideas, or look for an increase in papers on some approach Discontinuous jumps across a range of difficult benchmarks Could ask companies key questions like e.g. “are you still using English CoT?” and make them report Survey expectations at frontier labs (also covers intelligence explosion) Analysis attributing performance gains to different sources (e.g. model size scaleup, data quality improvement, RL, etc) Ryan: really big breakthroughs are like, AlexNet, GPT-1 / scale-up, scaling RL (o1, o3). Easy to notice something is turning into a big deal, hard to tell where on the Richter scale it will land. Could track the current 5 most promising potential trend-breakers. Ajeya: track breakdown of capabilities improvements over time: how much is from scaling pretraining, scaling RL posttraining, etc. Threshold effects / phase changes (e.g. the moment when people are no longer needed; maybe time horizon scaling suddenly becomes much easier) Phase changes in behavior of AI Village or other observations of AI capability / usage in real or realistic contexts Phase change in qualitative analysis of uplift trials Some resource (data, compute, electricity, intelligence headroom) is exhausted Note, may lead to workarounds rather than an end to progress. E.g. pretraining data gets harder to scale → scale RL instead. Hard to predict the effect. Low-hanging fruit is exhausted; training for long-horizon tasks is expensive Some important missing capability doesn’t yield to scaling + progress (e.g. sample-efficient learning, “judgement”, long-time-horizon skills) Economics of AI not working out → slowdown in investment AI winter – we run out of ideas for improving AI Josh: FRI expert panel. Could ask them “what’s the probability of an AI winter in the next 5 years”? External event (market downturn; war in Taiwan; public backlash / disaster → regulation; focused national effort / Manhattan Project) Track public sentiment and other political indicators Applications of models are a lagging indicator to model capabilities. First sign of a slowdown would be the time horizon graph slowing down. Other Nikola: if timelines are long enough to enable human emulations (Ems), that would make a difference, because we’d have been able to explore more implications. More generally, does other transformative tech come first. Possible techs: EMs, nanotech (unlikely), genetic engineering (seems like this could radically transform society in ~30 years if heavily invested in now, but in practice might not happen because of e.g. legal/regulatory blockers). === External funding for privately held AI companies raising above $1.5 million - Our World in Data (https://ourworldindata.org/grapher/private-investment-in-artificial-intelligence) === Our World in Data Browse by topic Data Insights Resources About Subscribe Donate Data External funding for privately held AI companies raising above $1.5 million See all data and research on: Artificial Intelligence Explore the Data Sources & Processing Reuse This Work Data sources and processing Insights about this data Data sources and processing Related articles Related charts Insights about this data Related articles Related charts What you should know about this indicator This data focuses on external private-market investment, such as venture-capital and private-equity deals. It does not include internal corporate R&D, capital expenditure (CapEx), or public-sector funding. Publicly traded companies, including large tech firms, are excluded. Because this data covers only one form of financing, it underestimates total global spending on AI. Large single deals can cause spikes in specific years. Broader economic conditions (interest rates, investor sentiment) can also drive changes that are not specific to AI. AI firms are identified by the source based on keywords and industry tags; some misclassification is possible. External funding for privately held AI companies raising above $1.5 million Money put into privately held AI companies that raised more than $1.5 million from private investors. This excludes publicly traded companies (e.g., public Big Tech companies) and companies’ own internal spending, such as R&D or infrastructure. Expressed in US dollars, adjusted for inflation. Source Quid via AI Index Report (2025); U.S. Bureau of Labor Statistics (2025) – with major processing by Our World in Data Last updated April 8, 2025 Next expected update April 2026 Date range 2013–2024 Unit constant 2021 US$ What you should know about this indicator This data focuses on external private-market investment, such as venture-capital and private-equity deals. It does not include internal corporate R&D, capital expenditure (CapEx), or public-sector funding. Publicly traded companies, including large tech firms, are excluded. Because this data covers only one form of financing, it underestimates total global spending on AI. Large single deals can cause spikes in specific years. Broader economic conditions (interest rates, investor sentiment) can also drive changes that are not specific to AI. AI firms are identified by the source based on keywords and industry tags; some misclassification is possible. External funding for privately held AI companies raising above $1.5 million Money put into privately held AI companies that raised more than $1.5 million from private investors. This excludes publicly traded companies (e.g., public Big Tech companies) and companies’ own internal spending, such as R&D or infrastructure. Expressed in US dollars, adjusted for inflation. Source Quid via AI Index Report (2025); U.S. Bureau of Labor Statistics (2025) – with major processing by Our World in Data Last updated April 8, 2025 Next expected update April 2026 Date range 2013–2024 Unit constant 2021 US$ Sources and processing This data is based on the following sources Quid via AI Index Report – AI Index Report The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence (AI). The mission is to provide unbiased, rigorously vetted, broadly sourced data to enable policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI. Retrieved on April 8, 2025 Retrieved from https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_2024_AI-Index-Report.pdf Citation This is the citation of the original data obtained from the source, prior to any processing or adaptation by Our World in Data. To cite data downloaded from this page, please use the suggested citation given in Reuse This Work below. Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, Tobi Walsh, Armin Hamrah, Lapo Santarlasci, Julia Betts Lotufo, Alexandra Rome, Andrew Shi, Sukrut Oak. “The AI Index 2025 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2025 The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence (AI). The mission is to provide unbiased, rigorously vetted, broadly sourced data to enable policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI. Retrieved on April 8, 2025 Retrieved from https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_2024_AI-Index-Report.pdf Citation This is the citation of the original data obtained from the source, prior to any processing or adaptation by Our World in Data. To cite data downloaded from this page, please use the suggested citation given in Reuse This Work below. Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, Tobi Walsh, Armin Hamrah, Lapo Santarlasci, Julia Betts Lotufo, Alexandra Rome, Andrew Shi, Sukrut Oak. “The AI Index 2025 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2025 U.S. Bureau of Labor Statistics – US consumer prices The Bureau of Labor Statistics reports the monthly Consumer Price Index (CPI) of individual goods and services for urban consumers at the national, city, and state levels. CPI is presented on an annual basis, which we have derived as the average of the monthly CPIs in a given year. Retrieved on April 12, 2025 Retrieved from https://www.bls.gov/data/tools.htm Citation This is the citation of the original data obtained from the source, prior to any processing or adaptation by Our World in Data. To cite data downloaded from this page, please use the suggested citation given in Reuse This Work below. U.S. Bureau of Labor Statistics The Bureau of Labor Statistics reports the monthly Consumer Price Index (CPI) of individual goods and services for urban consumers at the national, city, and state levels. CPI is presented on an annual basis, which we have derived as the average of the monthly CPIs in a given year. Retrieved on April 12, 2025 Retrieved from https://www.bls.gov/data/tools.htm Citation This is the citation of the original data obtained from the source, prior to any processing or adaptation by Our World in Data. To cite data downloaded from this page, please use the suggested citation given in Reuse This Work below. U.S. Bureau of Labor Statistics How we process data at Our World in Data All data and visualizations on Our World in Data rely on data sourced from one or several original data providers. Preparing this original data involves several processing steps. Depending on the data, this can include standardizing country names and world region definitions, converting units, calculating derived indicators such as per capita measures, as well as adding or adapting metadata such as the name or the description given to an indicator. At the link below you can find a detailed description of the structure of our data pipeline, including links to all the code used to prepare data across Our World in Data. Read about our data pipeline Notes on our processing step for this indicator Reporting a time series of AI investments in nominal prices would make it difficult to compare observations across time. To make these comparisons possible, one has to take into account that prices change (inflation). It is not obvious how to adjust this time series for inflation, and our team discussed the best solutions at our disposal. It would be straightforward to adjust the time series for price changes if we knew the prices of the specific goods and services purchased through these investments. This would make it possible to calculate a volume measure of AI investments and tell us how much these investments bought. But such a metric is not available. While a comprehensive price index is not available, we know that the cost of some crucial AI technology has fallen rapidly in price. In the absence of a comprehensive price index that captures the price of AI-specific goods and services, one has to rely on one of the available metrics for the price of a bundle of goods and services. Ultimately, we decided to use the US Consumer Price Index (CPI). The US CPI does not provide us with a volume measure of AI goods and services, but it does capture the opportunity costs of these investments. The inflation adjustment of this time series of AI investments, therefore, lets us understand the size of these investments relative to whatever else these sums of money could have purchased. Reuse this work All data produced by third-party providers and made available by Our World in Data are subject to the license terms from the original providers. Our work would not be possible without the data providers we rely on, so we ask you to always cite them appropriately (see below). This is crucial to allow data providers to continue doing their work, enhancing, maintaining and updating valuable data. All data, visualizations, and code produced by Our World in Data are completely open access under the Creative Commons BY license . You have the permission to use, distribute, and reproduce these in any medium, provided the source and authors are credited. Citations How to cite this page To cite this page overall, including any descriptions, FAQs or explanations of the data authored by Our World in Data, please use the following citation: “Data Page: External funding for privately held AI companies raising above $1.5 million”, part of the following publication: Charlie Giattino, Edouard Mathieu, Veronika Samborska, and Max Roser (2023) - “Artificial Intelligence”. Data adapted from Quid via AI Index Report, U.S. Bureau of Labor Statistics. Retrieved from https://archive.ourworldindata.org/20251119-141827/grapher/private-investment-in-artificial-intelligence.html [online resource] (archived on November 19, 2025). How to cite this data In-line citation If you have limited space (e.g. in data visualizations), you can use this abbreviated in-line citation: Quid via AI Index Report (2025); U.S. Bureau of Labor Statistics (2025) – with major processing by Our World in Data Full citation Quid via AI Index Report (2025); U.S. Bureau of Labor Statistics (2025) – with major processing by Our World in Data. “External funding for privately held AI companies raising above $1.5 million” [dataset]. Quid via AI Index Report, “AI Index Report”; U.S. Bureau of Labor Statistics, “US consumer prices” [original data]. Retrieved January 8, 2026 from https://archive.ourworldindata.org/20251119-141827/grapher/private-investment-in-artificial-intelligence.html (archived on November 19, 2025). Our World in Data is free and accessible for everyone. Help us do this work by making a donation. Donate now Our World in Data is a project of Global Change Data Lab , a nonprofit based in the UK (Reg. Charity No. 1186433). Our charts, articles, and data are licensed under CC BY , unless stated otherwise. Tools and software we develop are open source under the MIT license . Third-party materials, including some charts and data, are subject to third-party licenses. See our FAQs for more details. Explore Topics Data Insights Resources Latest SDG Tracker Teaching with OWID About About Us Organization Funding Team Jobs FAQs RSS Feeds Research & Writing Data Insights Follow us Privacy policy Legal disclaimer Grapher license === The MIT AI Risk Repository (https://airisk.mit.edu/) === Risks IncidentS Mitigations Governance Blog TEAM What are the risks of Artificial Intelligence? A comprehensive living database of over 1700 AI risks categorized by their cause and risk domain Get updates Explore database What is the AI Risk Repository? The AI Risk Repository has three parts: The AI Risk Database captures 1700+ risks extracted from 74 existing frameworks and classifications of AI risks The Causal Taxonomy of AI Risks classifies how, when, and why these risks occur The Domain Taxonomy of AI Risks classifies these risks into 7 domains and 24 subdomains (e.g., “False or misleading information”) The repository is part of the MIT AI Risk Initiative, which aims to increase awareness and adoption of best practice AI risk management across the AI ecosystem. How can I use the Repository? The AI Risk Repository provides: An accessible overview of threats from AI A regularly updated source of information about new risks and research A common frame of reference for researchers, developers, businesses, evaluators, auditors, policymakers, and regulators A resource to help develop research, curricula, audits, and policy An easy way to find relevant risks and research AI Risk Database The AI Risk Database links each risk to the source information (paper title, authors), supporting evidence (quotes, page numbers), and to our Causal and Domain Taxonomies. You can experiment with a preview version of the database in the embed below, or copy the full database on Google Sheets , or OneDrive . Watch our explainer video on YouTube for a walkthrough of the database and how to use it. Causal Taxonomy of AI Risks The Causal Taxonomy of AI risks classifies how, when, and why an AI risk occurs. View the Causal Taxonomy on a single page Read our research report for more detail on how the Taxonomy was constructed and what it reveals about risks from AI Explore the taxonomy in the figure below Entity AI : Due to a decision or action made by an AI system Human : Due to a decision or action made by humans Other : Due to some other reason or ambiguous Intent Intentional : Due to an expected outcome from pursuing a goal Unintentional : Due to an unexpected outcome from pursuing a goal Other : Without clearly specifying the intentionality Timing Pre-deployment : Before the AI is deployed Post-deployment : After the AI model has been trained and deployed Other : Without a clearly specified time of occurrence Get a quick preview of how we group risks by causal factors in our database. Search for one of the causal factors (eg 'pre-deployment') to see all risks categorized against that factor. For more detailed filtering and to freely download the data, explore the full database . Domain Taxonomy of AI Risks The Domain Taxonomy of AI Risks classifies risks from AI into seven domains and 24 subdomains. View the Domain Taxonomy on a single page Read our research report for more detail on how the Taxonomy was constructed and what it reveals about risks from AI Explore the taxonomy in the interactive figure below 1. Discrimination & Toxicity Risks related to unfair treatment, harmful content exposure, and unequal AI performance across different groups and individuals. 1.1 Unfair discrimination and misrepresentation Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and representation of those groups. 1.2 Exposure to toxic content AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography. 1.3 Unequal performance across groups Accuracy and effectiveness of AI decisions and actions is dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users. 2. Privacy & Security Risks related to unauthorized access to sensitive information and vulnerabilities in AI systems that can be exploited by malicious actors. 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or loss of confidential intellectual property. 2.2 AI system security vulnerabilities and attacks Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior. 3. Misinformation Risks related to AI systems generating or spreading false information that can mislead users and undermine shared understanding of reality. 3.1 False or misleading information AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms. 3.2 Pollution of information ecosystem and loss of consensus reality Highly personalized AI-generated misinformation creating "filter bubbles" where individuals only see what matches their existing beliefs, undermining shared reality, weakening social cohesion and political processes. 4. Malicious Actors Risks related to intentional misuse of AI systems by bad actors for harmful purposes including disinformation, cyberattacks, and fraud. 4.1 Disinformation, surveillance, and influence at scale Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior. 4.2 Fraud, scams, and targeted manipulation Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery. 4.3 Cyberattacks, weapons development or use and mass harm Using AI systems to develop cyber weapons (e.g., coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or CBRNE), or use weapons to cause mass harm. 5. Human-Computer Interaction Risks related to problematic relationships between humans and AI systems, including overreliance and loss of human agency. 5.1 Overreliance and unsafe use Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties. 5.2 Loss of human agency and autonomy Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled. 6. Socioeconomic & Environmental Risks related to AI's impact on society, economy, governance, and the environment, including inequality and resource concentration. 6.1 Power centralization and unfair distribution of benefits AI-driven concentration of power and resources within certain entities or groups, especially those with access to or ownership of powerful AI systems, leading to inequitable distribution of benefits and increased societal inequality. 6.2 Increased inequality and decline in employment quality Widespread use of AI increasing social and economic inequalities, such as by automating jobs, reducing the quality of employment, or producing exploitative dependencies between workers and their employers. 6.3 Economic and cultural devaluation of human effort AI systems capable of creating economic or cultural value, including through reproduction of human innovation or creativity (e.g., art, music, writing, code, invention), can destabilize economic and social systems that rely on human effort. This may lead to reduced appreciation for human skills, disruption of creative and knowledge-based industries, and homogenization of cultural experiences due to the ubiquity of AI-generated content. 6.4 Competitive dynamics AI developers or state-like actors competing in an AI 'race' by rapidly developing, deploying, and applying AI systems to maximize strategic or economic advantage, increasing the risk they release unsafe and error-prone systems. 6.5 Governance failure Inadequate regulatory frameworks and oversight mechanisms failing to keep pace with AI development, leading to ineffective governance and the inability to manage AI risks appropriately. 6.6 Environmental harm The development and operation of AI systems causing environmental harm, such as through energy consumption of data centers, or material and carbon footprints associated with AI hardware. 7. AI System Safety, Failures, & Limitations Risks related to AI systems that fail to operate safely, pursue misaligned goals, lack robustness, or possess dangerous capabilities. 7.1 AI pursuing its own goals in conflict with human goals or values AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals. 7.2 AI possessing dangerous capabilities AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system. 7.3 Lack of capability or robustness AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning. 7.4 Lack of transparency or interpretability Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors. 7.5 AI welfare and rights Ethical considerations regarding the treatment of potentially sentient AI entities, including discussions around their potential rights and welfare, particularly as AI systems become more advanced and autonomous. 7.6 Multi-agent risks Risks from multi-agent interactions, due to incentives (which can lead to conflict or collusion) and/or the structure of multi-agent systems, which can create cascading failures, selection pressures, new security vulnerabilities, and a lack of shared information and trust. Get a quick preview of how we group risks by domain in our database. Search for one of the domain/subdomain names (eg 'fraud') to see all risks categorized against that domain. For more detailed filtering and to freely download the data, explore the full database . How to use the AI Risk Repository Our Database is free to copy and use The Causal and Domain Taxonomies can be used separately to filter this database to identify specific risks, for instance, risks occurring pre-deployment or post-deployment or related to Misinformation The Causal and Domain Taxonomies can be used together to understand how each causal factor (i.e., entity , intention and timing ) relate to each risk domain. For example, to identify the intentional and unintentional variations of Discrimination & toxicity Offer feedback or suggest missing resources, or risks, here , or email airisk[at]mit.edu We provide examples of use cases for some key audiences below. How policymakers might use this tool To understand the research and policy landscape. For risk assessments to inform policy decisions. As shared framework for discussing AI risks with other groups. As a way to monitor emergent risks and ensure complete oversight. To identify new, previously undocumented risks. To prioritize and plan funding. How risk evaluators might use this tool To identify new, previously undocumented, risks. To understand the risk landscape and curate or create related evaluations. As a framework for discussing risks and potential evaluations with clients. As a basis for developing specific risk determination criteria. As a way to determine and communicate the scope of an audit. How academics might use this tool As a foundation for developing other classifications (e.g., the actions taken to address specific types of risks, or the actors involved in those risks). To find underexplored areas of AI risk research. To develop material for education and training. To help validate where they have identified new, previously undocumented, risks. To understand the landscape of existing research. How industry might use this tool To conduct internal risk assessments. To identify new, previously undocumented, risks. Evaluating risk exposure and developing risk mitigation strategies. To develop research and training. Frequently Asked Questions How can I access the database without a Google account? Please access via OneDrive . We will create a better formatted version or find a better solution in the future. How did you create the AI Risk Repository? We used a systematic search strategy, forwards and backwards searching, and expert consultation, to identify 65 AI risk classifications, frameworks, and taxonomies. We extracted 1600+ risks from these documents into a living AI risk database. We used the best fit framework synthesis approach to create our taxonomies. This involved selecting an existing framework, coding extracted risks against it, then iteratively refining the framework through analysis of risks until we developed a comprehensive structure that could effectively categorize all relevant risks. Which existing frameworks & documents did you include? You can view the frameworks that were identified and extracted into the Repository: As a slide deck , showing key figures or tables presenting the framework, and citation information for the document. In the database , with metadata and extracted information for each document. What can I do if I think there is a missing risk or resource? Use this form to offer feedback, suggest resources or risks, or make contact. You can also email pslat[at]mit.edu. What are some limitations of the current AI Risk Repository? The Repository has several limitations: Limited to risks from 65 documents (although we screened >17,000 records after a systematic search of peer-reviewed and gray literature) May be missing emerging, domain-specific risks, and unpublished risks. Has potential for errors and subjective bias; we used a single expert reviewer for extraction and coding. May include poorly communicated or unclear risks: we extracted risks as presented. Our taxonomies prioritize clarity and simplicity over nuance. Our taxonomies do not categorize risks by potentially important factors such as risk impact, likelihood, or discuss the interaction between risks. See our for a full list and suggestions for future research. Why do you have two taxonomies? During this synthesis process, we realized that our database broadly contained two types of classification systems: High-level categorizations of causes of AI risks (e.g., when or why risks from AI occur) Mid-level hazards or harms from AI (e.g, AI is trained on limited data or used to make weapons) Because these classification systems were so different, it was hard to unify them; high-level risk categories such as “Diffusion of responsibility” or “Humans create dangerous AI by mistake” do not map to narrower categories like “Misuse” or “Noisy Training Data,” or vice versa. We therefore decided to create two different classification systems that together would form our unified classification system. Is this unique? To the best of our knowledge, this is the first comprehensive review of AI risk frameworks and taxonomies which extracts their risks and releases that data for further adaptation and use. Please let us know of anything that we may have missed. What are some other databases of AI risks? https://attack.mitre.org/ https://airisk.io/ https://avidml.org/#efforts https://www.aitracker.org/#catalog-tabs Please let us know of anything that we may have missed. How do I cite the AI Risk Repository? To reference our repository, you can cite our pre-print paper: Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2024). The AI Risk Repository: A Comprehensive Meta-Review, Database, and Taxonomy of Risks from Artificial Intelligence . https://doi.org/10.48550/arXiv.2408.12622 Team Peter Slattery MIT FutureTech Alexander Saeri MIT FutureTech & The University of Queensland Michael Noetel MIT FutureTech & The University of Queensland Jess Graham MIT FutureTech & The University of Queensland Neil Thompson MIT FutureTech Alumni Emily Grundy MIT FutureTech Stephen Casper MIT Computer Science and Artificial Intelligence Laboratory Soroush Pour Harmony Intelligence Risto Uuk Future of Life Institute & KU Leuven James Dao Harmony Intelligence Acknowledgments Feedback and useful input: Anka Reuel, Michael Aird, Greg Sadler, Matthjis Maas, Shahar Avin, Taniel Yusef, Elizabeth Cooper, Dane Sherburn, Noemi Dreksler, Uma Kalkar, CSER, GovAI, Nathan Sherburn, Andrew Lucas, Jacinto Estima, Kevin Klyman, Bernd W. Wirtz, Andrew Critch, Lambert Hogenhout, Zhexin Zhang, Ian Eisenberg, Stuart Russell, and Samuel Salzer . Risks Incidents Mitigations Governance Blog Team © MIT FutureTech 2025 The MIT AI Risk Initiative is licensed under CC BY 4.0 MIT Accessibility === Welcome to the Artificial Intelligence Incident Database (https://incidentdatabase.ai/) === Skip to Content AI Incident Database Open Twitter Open RSS Feed Open Facebook Open LinkedIn Open GitHub Open Menu Search Discover Submit Welcome to the AIID Discover Incidents Spatial View Table View List view Entities Taxonomies Submit Incident Reports Submission Leaderboard Blog AI News Digest Risk Checklists Random Incident Sign Up Collapse Discover Submit Welcome to the AIID Discover Incidents Spatial View Table View List view Entities Taxonomies Submit Incident Reports Submission Leaderboard Blog AI News Digest Risk Checklists Random Incident Sign Up Collapse Welcome to the AI Incident Database Search Discover Loading... Incident 1323: Madhya Pradesh Congress Alleges AI-Generated Images Were Submitted in National Water Award Process “ ‘Smart corruption’: Congress claims AI images used to bag national water award; MP administration rejects charge ” Latest Incident Report hindustantimes.com 2025-12-30 The Congress on Monday accused the Madhya Pradesh government of corruption, alleging that the Khandwa district administration secured a national water conservation award by submitting AI-generated images, a charge denied by the district's officials. MP water award controversy: Congress alleges misuse of AI, officials call claims false(X/@jitupatwari) The allegations came after Khandwa district secured first place at the national level for water conservation efforts under the Centre's Jal Sanchay, Jan Bhagidari campaign. According to news agency PTI, the district went on to receive a ₹2-crore award at the sixth National Water Awards ceremony held in New Delhi in November. The Kaveshwar gram panchayat in the district also won second prize in the best gram panchayat category. Congress alleges use of AI-generated images Madhya Pradesh Congress president Jitu Patwari raised the allegation in a post on X, accusing the BJP-led state government of misusing artificial intelligence to project false development. "Where the BJP government should teach our children the proper use of AI, it is itself indulging in corruption using AI," he added. "In Khandwa, officials of the BJP government turned two-foot-deep pits into wells using AI, and uploaded AI-generated images of various development works across the area on the portal," Patwari alleged. "Based on these images, they even took an award from the Honourable President," he claimed. He further said, "When the ground reality came to light, fields and empty grounds were found there. Clearly, this was not water conservation, but a game of technology-created images. Under the BJP rule, corruption has also become smart." District administration rejects charge As the controversy escalated, the Khandwa district administration held a press conference to refute the allegations. District panchayat chief executive officer Nagarjun B Gowda said that AI-generated images had no connection with the National Water Award process, as reported by news agency PTI. He said "verified images after thorough scrutiny" of 1,29,046 water conservation works carried out under the Jal Sanchay, Jan Bhagidari campaign were uploaded on the official JSJB portal. According to Gowda, the Union jal shakti ministry verified all uploaded material and conducted random field inspections of one per cent of the total works. "Prima facie, false news about water conservation works carried out in Khandwa district is being spread by some social media accounts," he said, reported PTI. Separate portal, limited AI images The CEO clarified that photographs related to water conservation are also uploaded on another platform -- the Catch the Rain portal -- but only for educational and motivational purposes. "The district administration has found that 21 images generated through AI were uploaded on the Catch the Rain portal. This was possibly done with malicious intent. The district administration is taking action against those who uploaded these images," the news agency quoted him Gowda as saying. He stressed that the Catch the Rain portal is separate from the Jal Sanchay, Jan Bhagidari campaign portal. "The Catch the Rain portal is completely different from the Jal Sanchay, Jan Bhagidari campaign portal. Awards under the Jal Sanchay, Jan Bhagidari campaign are not considered based on images uploaded on the Catch the Rain portal," he added, PTI reported. More than 1.25 lakh water conservation works were carried out in Khandwa district under the Jal Sanchay, Jan Bhagidari campaign, the highest number recorded in the country, the official said. Read More Loading... Incident 1317: Purported Deepfake Impersonation of Elon Musk Used to Promote Fraudulent '17-Hour' Diabetes Treatment Claims “ Boosie Badazz Fooled by AI-Deepfake: Elon Musk's Nonexistent 17-Hour Diabetes Cure ” opentools.ai 2025-12-28 In a bizarre twist, rapper Boosie Badazz fell for an AI-deepfake video of Elon Musk promoting a fake diabetes cure, showing both the dangers of AI technology and the pitfalls of misinformation. Learn why this scam caught attention, its risks, and public reactions. Introduction: The Viral Deepfake Incident Involving Boosie Badazz and Elon Musk The incident involving Boosie Badazz and a deepfake video of Elon Musk serves as a compelling introduction to the challenges posed by AI-generated content in today's digital landscape. Boosie Badazz, a well-known rapper, unknowingly amplified a fabricated video in which Musk purportedly endorsed a miraculous '17-hour' diabetes cure. This incident not only highlights the perils of sophisticated technology but also underscores the vulnerability of public figures to misinformation, especially when it pertains to serious health issues. This viral deepfake quickly spread across social media platforms, drawing significant attention and reactions from the public and media alike. According to Complex , the video was entirely fabricated and designed to deceive viewers into purchasing unverified health products. Celebrities like Boosie Badazz, who have substantial social media influence, can inadvertently contribute to the spread of such misinformation, thus complicating efforts to educate the public about the realities of managing chronic conditions like diabetes. The choice of Elon Musk as the figure in this deepfake is telling, as his name carries credibility and authority in technology and innovation. By associating Musk with a fictitious health product, the creators of the deepfake cleverly exploited the trust many have in him as a public figure. This incident serves as a stark reminder of the increasing sophistication of AI tools that can not only imitate visual and vocal patterns accurately but also manipulate public perception and trust. The case of Boosie Badazz and the deepfake video illustrates broader issues within the realm of AI-generated misinformation. As AI technologies continue to advance, the potential for misuse in various domains, including health, becomes more pronounced. This particular incident acts as a catalyst for ongoing discussions about the ethical and regulatory frameworks needed to combat digital deceit effectively. The spread of such misinformation not only endangers individuals who might fall prey to false health claims but also poses significant challenges to public health communication efforts. Analyzing the Nature and Origins of the Deepfake Video The phenomenon of deepfake videos has seen a substantial rise in recent years, posing significant challenges in the realms of both technology and information authenticity. One such instance involves a viral video falsely depicting Elon Musk endorsing a "17-hour" diabetes cure. This video, which was created using sophisticated AI technologies, was a prime example of how deepfakes can be engineered to spread misinformation under the guise of credibility. The real danger lies in their ability to exploit the influence of high-profile individuals to lend false credibility to unauthorized claims and advertisements. The Musk video, as outlined in a report from Complex , not only deceived viewers who might have been desperate for a medical breakthrough but also highlighted the pressing issue of health misinformation proliferating online. Deepfake technology, while a marvel of modern AI development, becomes a tool of deception when misused to concoct fraudulent endorsements. This can be especially perilous in the healthcare sector, where misinformation can lead to significant harm. The use of a deepfake to allegedly portray Elon Musk as a proponent of an instantaneous diabetes cure illuminates the ease with which false narratives can be constructed and disseminated. Despite the rapid debunking by AI verification tools like Grok, and widespread disapproval from media outlets, the video managed to gain traction, demonstrating the power of deepfakes to subvert trust and amplify falsehoods in the digital age, as reported in various news articles including Complex . The origins of deepfakes can be traced back to the evolution of AI in generating highly realistic audio and visual content. Initially, these technologies were celebrated for their potential to revolutionize digital content creation. However, their application in creating misleading videos, such as the one involving Elon Musk, underscores the ethical quandaries and challenges they present. The viral deepfake video claiming Musk's involvement with a diabetes cure not only fabricates a falsehood but also preys on the vulnerable who may be in critical need of legitimate medical information. Instances such as these, reported by Complex , stress the imperative need for more robust detection technologies and stricter regulatory frameworks to mitigate the misuse of deepfake technologies in public and health-related contexts. Public and Media Reactions to the Viral Video The internet buzzed with a mixture of fascination and skepticism when rapper Boosie Badazz reacted to a viral video featuring what appeared to be Elon Musk endorsing a revolutionary diabetes cure. Social media was flooded with memes and comments, with many users mocking Boosie for believing the deepfake and for sharing his assistant's phone number seeking more information. These reactions highlighted the rapidly spreading misinformation and the naivety of even public figures when encountering such deceptive content as reported by Complex . The public discourse following the viral video mostly centered around the cautionary tale of Boosie's reaction. On platforms like Twitter, users and commentators voiced their opinions, with some emphasizing the need to educate oneself about such scams. Others expressed sympathy for Boosie, recognizing that desperation for health solutions can cloud judgment, especially when it involves serious conditions like diabetes. This incident also sparked conversations about the need for better digital literacy to combat the spread of AI-generated misinformation as noted in related commentary . Media outlets and fact-checking organizations quickly moved to label the viral video as a deepfake scam, aimed at selling unverified supplements rather than offering genuine medical advice. This swift response showcased the vital role of media in debunking false information and providing clarification to the public. However, the incident also underscored the persistent challenge of controlling digital misinformation, as platforms struggled to catch up with the rapid dissemination of fake news, amplified by the involvement of prominent personalities like Boosie according to Complex . Debunking the Fake: Verification and Fact-Checking Efforts In the rapidly evolving digital landscape, the vital role of verification and fact-checking efforts cannot be overstated, particularly in debunking fake content such as AI-generated deepfakes. The misleading video falsely attributing a "17-hour diabetes cure" endorsement to Elon Musk represents a significant challenge in the digital ecosystem. As noted in a Complex article , this deepfake was flagged accurately by AI tools and numerous media outlets, exposing its fraudulent nature. This scenario underscores the critical need for advanced verification processes to combat the propagation of such menacing digital fabrications. Effective verification and fact-checking are crucial when dealing with the spread of misinformation, particularly in the realm of health-related content. In the case of the illusory Elon Musk diabetes cure clip, scrutinizing the video's production quality and cross-referencing public records were key strategies employed by AI tools to identify the scam. Additionally, reputable media outlets played a vital role by providing contextual insights that further debunked the clip. Such efforts illustrate the importance of a coordinated approach when dismantling misleading content across platforms. The incident involving the deceptive Elon Musk deepfake demonstrates the ongoing threat posed by sophisticated AI-generated content and highlights the importance of immediate response measures by verification teams. AI tools such as Grok were instrumental in flagging the false advertisement for what it truly was---a scam. With high-profile figures like Musk being impersonated, it becomes alarmingly clear how deepfakes can blur lines between reality and illusion. It is imperative for stakeholders in both technology and media sectors to enhance collaborative verification strategies to keep pace with the evolving digital threats. The verification response to the rapid spread of the fake Elon Musk video illustrates a significant societal challenge: ensuring the authenticity of content shared across digital platforms. As fake content becomes increasingly sophisticated, consumer trust hinges more than ever on the robustness of fact-checking mechanisms. As detailed in the Complex article , various sources effectively used AI tools to unmask the scam, providing a critical line of defense against misinformation. The collaborative effort between media and technology firms exemplifies a proactive stance necessary to safeguard informational integrity. In the wake of the fake Elon Musk diabetes video, the role of fact-checkers has become more relevant than ever. Through rigorous analysis and the deployment of advanced AI technologies, teams were able to quickly identify and expose the video as fraudulent. This highlights the pressing need for continuous enhancements in verification technology to adapt to the cunning tactics of bad actors in the digital space. The experience gained from tackling such misinformation feeds into strategic planning for future occurrences, emphasizing the shared responsibility among digital stakeholders to maintain public trust. Understanding the Health Risks: The Absence of a 17-Hour Diabetes Cure The claim of a '17-hour diabetes cure,' as purported in a viral AI deepfake video, poses significant health risks due to its misleading nature. According to the original source , this video falsely represented Elon Musk promoting a rapid cure, which many might be tempted to believe due to the trust placed in his public persona. This kind of misinformation is particularly dangerous as it exploits the vulnerabilities of those with chronic conditions like diabetes, potentially delaying effective treatment. The false premise of an instant cure undermines the reality of diabetes management, which requires continuous medical treatment and lifestyle changes. The absence of a genuine cure for diabetes, especially within a day, is well documented within the medical community. According to health experts, type 1 diabetes necessitates lifelong insulin therapy, while type 2 diabetes management involves sustained lifestyle modifications and medical supervision. The circulation of such deceptive information, as seen with the viral clip, serves only to create false hope and distract from legitimate treatment paths. This incident with the deepfake video also highlights broader concerns about the misuse of AI technology in spreading health misinformation. The potential of AI-generated content to mimic credible figures like Elon Musk can inadvertently convince people to pursue ineffective or harmful products. This trend is not only a public health risk but also a legal and ethical issue for platforms hosting such content. Fact-checkers and AI tools have already identified the video as a scam, underscoring the urgency for improved detection and prevention measures to protect consumers from fraudulent health claims. Social media platforms are at the forefront of this battle, where false information spreads rapidly, necessitating robust verification systems. The Elon Musk deepfake case should serve as a wake-up call to strengthen these systems and educate users about the realities of diabetes treatment and the dangers of miracle cures. Adopting a skeptical approach towards sensational health claims and consulting healthcare professionals is pivotal in safeguarding against scams, thus promoting a more informed and health-conscious society. Broader Implications of Deepfakes in Health Misinformation The rise of AI-generated deepfakes presents substantial challenges, particularly in the domain of health misinformation. As demonstrated by a viral video falsely depicting Elon Musk promoting a rapid '17-hour' diabetes cure, deepfakes have the power to significantly mislead the public by leveraging the influence of trusted figures. These fabrications not only create panic but also erode trust in legitimate medical advice, often targeting vulnerable individuals desperately seeking solutions for their chronic health conditions. According to Complex's article , Boosie Badazz's public reaction to the fake video highlights how easily misinformation can spread when amplified by a public figure. This scenario underscores the pressing need for increased public awareness and robust verification mechanisms to counteract such deceptive practices. Moreover, deepfakes in health misinformation carry broader implications that extend beyond the immediate deception of individuals into societal and economic spheres. The financial impact is substantial, with fraudulent supplements and miracle cures potentially driving significant profits for scammers, thereby exploiting consumer vulnerability. The Complex article notes how AI-generated fraud could cost the global economy billions annually. Additionally, the psychological toll on individuals who fall prey to such scams, believing in miraculous cures, results in further harm by delaying appropriate medical treatment and contributing to health crises. The spreading of such deepfakes also risks promoting a culture of skepticism towards genuine medical breakthroughs, thus hindering healthcare advancements and undermining efforts to communicate effective public health strategies. Steps Towards Prevention: Legal and Technological Solutions The rapid proliferation of AI-generated deepfakes, especially in sensitive fields like health, necessitates a multifaceted approach to prevention and mitigation. Legal frameworks need immediate updates to catch up with technological advancements. Currently, many jurisdictions lack specific laws targeting the creation and dissemination of AI-generated misinformation, including deepfakes. This legal gap allows perpetrators to exploit technology without fearing legal repercussions. As highlighted in recent cases , these scams can have serious consequences for individuals seeking genuine medical solutions, making it imperative for lawmakers to act swiftly. In addition to legal measures, technological solutions are crucial in combating deepfake-related health scams. Advances in artificial intelligence are not only a challenge but also a tool in developing prevention strategies. Companies and research institutions are investing in AI technologies to detect and label synthetic media. For instance, fact-checking tools are employed by platforms to swiftly identify and remove fraudulent content. There is a significant push for implementing watermarking technologies that can verify the authenticity of digital media. As these solutions grow more sophisticated, they will become an essential part of the toolkit used by social media platforms and regulatory agencies. Furthermore, public awareness and education are fundamental components of preventing deepfake scams. Users need to be informed about the nature of deepfakes and how to critically assess content they encounter on digital platforms. Efforts like promoting 'AI literacy' are integral to equipping the public with skills to discern and challenge dubious health claims. The incident involving Boosie Badazz underscores the potential for widespread dissemination of AI-driven scams when prominent figures inadvertently validate such content. As detailed in various discussions , strengthening digital literacy alongside legal and technological advancements could form a robust shield against the misuse of AI in health misinformation. Conclusion: Lessons Learned and Future Outlook The controversy around an AI-generated deepfake involving Elon Musk and a fictitious diabetes cure underscores urgent lessons in our digital landscape. This incident highlighted the susceptibility of even influential personalities, such as rapper Boosie Badazz, to sophisticated scams that manipulate deepfakes for fraudulent ends. According to Complex , Boosie's reaction to the video, believing it to be authentic, demonstrates the persuasive power of deepfakes and the challenges they pose in discerning genuine content from fabricated lies. This teaches a crucial lesson in skepticism and the necessity for personal vigilance in verifying information, especially concerning health claims. Read More Loading... Incident 1318: School's Suspected AI-Cheating Allegation Precedes Student's Reported Suicide in Greater Noida, India “ 'Questioned if She Used AI': Greater Noida Teen Dies by Suicide After School Speculated Cheating in Exams ” timesnownews.com 2025-12-28 **Noida: **A 16-year-old Class 10 student allegedly died by suicide after being questioned by her school authorities over the suspected use of AI-based assistance during a pre-board examination, police said on Saturday (December 27). The incident occurred on December 23 in Greater Noida West, Uttar Pradesh. According to the police, the student was confronted by teachers and the school principal after her mobile phone was allegedly found being used to access AI tools during the exam. The girl's father has filed a complaint accusing the principal and teachers of mental harassment and abetment, alleging she was humiliated publicly, causing severe emotional distress. The school denied the allegations, stating the phone was confiscated and the student was reprimanded strictly in line with CBSE examination rules. The principal said the interaction was brief and non-abusive. Police confirmed that CCTV footage has been submitted by the school and the matter is under investigation. Meanwhile, a Times of India report added that the father alleged that his daughter was deeply distressed after being scolded by her teachers for bringing her mobile phone to the exam hall on December 22. According to the father, he has three daughters who all study at the same school. He said his eldest daughter had **"**unknowingly" brought her phone to school on the day of the exam. The invigilator caught her with the phone, reprimanded her, and informed her class teacher. She was then taken to the principal. The father alleged that even after he arrived at the school, the teachers and principal continued to scold and insult his daughter harshly. He claimed that the teachers called him "careless" and that their aggressive behaviour had a serious psychological impact on his daughter. He said the incident had also left his other two daughters traumatised and afraid to return to school. In his complaint, the father named two teachers, and the school management. He asked the police to register a case under Section 108 of the BNS (abetment of suicide) and other relevant laws. The school, however, has strongly denied the allegations. The principal told TOI that the student was not harassed and that the school followed the standard protocols prescribed by the Central Board of Secondary Education (CBSE). She added that the child did not cry or show signs of distress at the time. The school also has CCTV footage of the entire incident, which has been handed over to the police. The principal further said that the school encourages students to manage academic stress through activities like dance performances, and highlighted that the girl had recently participated in the school's annual day event. The police are currently investigating the matter, taking statements from the family, school authorities, and other witnesses, and reviewing all available evidence, including CCTV footage. Read More Loading... Incident 1319: Purported Deepfake Investment Video Reportedly Used in Scam That Defrauded Turkish Couple of 1.5 Million Lira (~$35,000 USD) “ Artificial intelligence defrauded people of 1.5 million Turkish Lira! ” yeniakit.com.tr 2025-12-28 A married couple living in the İlkadım district of Samsun have filed a criminal complaint with the prosecutor's office, alleging they were defrauded of 1.5 million Turkish Lira after believing an investment video created using artificial intelligence. İsa Kereci, 53, a scrap metal collector residing in the Kadıköy neighborhood of İlkadım district, and his wife, Hale Kereci, 48, a mother of four, explained that they fell into the scammers' trap after clicking on an advertisement they found on Facebook. The couple stated that in the so-called investment process, which initially started with small amounts, they were misled with justifications such as "file fees," "insurance costs," "tax debt," and "Central Bank transfers." They further stated that they were made to share their screens, resulting in money transfers from their bank accounts and credit cards to different individuals. İsa Kereci and Hale Kereci stated that during this process, they were made to take out uninsured loans, their savings in their savings account were withdrawn, purchases were made from their credit cards, they sold their gold, and borrowed money from relatives, resulting in a total loss of approximately 1.5 million Turkish Lira for the fraudsters. The couple, who went to the Samsun Courthouse and filed a complaint with the prosecutor's office after the incident, warned citizens to be cautious of similar advertisements. Hale Kereci shed tears in front of the courthouse. (The text ends abruptly here, likely due to a copy-paste error.) Read More Loading... Incident 1320: Purportedly AI-Manipulated Image Reported to Falsely Depict Taiwanese Politician Kao Chia-yu Posing With PRC Flag “ Kao Chia-yu went to China to take a photo with the Chinese national flag? She angrily issued a statement clarifying: She will not go to China under three conditions. ” nownews.com 2025-12-28 Just recently, unscrupulous individuals used AI-generated video to forge Wang Shih-chien's identity, claiming he had traveled to China. Former legislator Kao Chia-yu has also become a victim. Someone maliciously photoshopped a picture of her at the Taipei Economic and Cultural Office in New York, replacing the Republic of China flag in the background with the Chinese national flag. Kao immediately clarified on Facebook, urging the public to be vigilant and not be misled by such low-quality fake news. Kao Chia-yu posted a comparison picture on Facebook, showing that the original Republic of China flag had been replaced with the Chinese national flag, a malicious attempt to smear Taiwan. Kao firmly stated her position, outlining three prerequisites for traveling to China. She solemnly declared that she would absolutely not set foot on Chinese territory until China released Jimmy Lai, Santa Claus appeared in Shanghai, and she "recognized the Republic of China (Taiwan)." These remarks directly refuted the fake news and demonstrated her firm stance in safeguarding Taiwan's sovereignty. Some observant netizens even spotted flaws in the altered images, commenting, "They just love photoshopping; they didn't even bother to remove the plum blossoms on the wall. So unprofessional, haha." Others left encouraging comments, such as, "This is really too exaggerated," and "They've really been completely infiltrated." As the 2026 local elections approach, composite images and misinformation targeting politicians are frequently circulating online. The public should verify information from multiple sources when receiving it online to avoid being misled. Read More Quick Add New Report URL Submit Submitted links are added to a review queue to be resolved to a new or existing incident record. Incidents submitted with full details are processed before URLs not possessing the full details. About the Database The AI Incident Database is dedicated to indexing the collective history of harms or near harms realized in the real world by the deployment of artificial intelligence systems. Like similar databases in aviation and computer security, the AI Incident Database aims to learn from experience so we can prevent or mitigate bad outcomes. You are invited to submit incident reports, whereupon submissions will be indexed and made discoverable to the world. Artificial intelligence will only be a benefit to people and society if we collectively record and learn from its failings. (Learn More) AI Incident Roundup – August, September, and October 2025 By Daniel Atherton 2025-11-08 At Templestowe, Arthur Streeton, 1889 🗄 Trending in the AIID Across August, September, and October 2025, the AI Incident Database logged one ... Read More The Database in Print Read about the database on the PAI Blog , Vice News , Venture Beat , Wired , arXiv , and Newsweek among other outlets. Incident Report Submission Leaderboards These are the persons and entities credited with creating and submitted incident reports. More details are available on the leaderboard page. New Incidents Contributed 🥇 Daniel Atherton 680 🥈 Anonymous 154 🥉 Khoa Lam 93 Reports added to Existing Incidents 🥇 Daniel Atherton 771 🥈 Anonymous 231 🥉 Khoa Lam 230 Total Report Contributions 🥇 Daniel Atherton 2955 🥈 Anonymous 968 🥉 1 587 The AI Incident Briefing Create an account to subscribe to new incident notifications and other updates. Random Incidents Loading... Predictive Policing Biases of PredPol Loading... Aledo High School Student Allegedly Generates and Distributes Deepfake Nudes of Seven Female Classmates Loading... ChatGPT Reportedly Introduces Errors in Critical Child Protection Court Report Loading... Child Sexual Abuse Material Taints Image Generators Loading... ChatGPT Reportedly Produced False Court Case Law Presented by Legal Counsel in Court The Responsible AI Collaborative The AI Incident Database is a project of the Responsible AI Collaborative, an organization chartered to advance the AI Incident Database. The governance of the Collaborative is architected around the participation in its impact programming. For more details, we invite you to read the founding report and learn more on our board and contributors . View the Responsible AI Collaborative's Form 990 and tax-exempt application. Organization Founding Sponsor Database Founding Sponsor Sponsors and Grants In-Kind Sponsors Research Defining an “AI Incident” Defining an “AI Incident Response” Database Roadmap Related Work Download Complete Database Project and Community About Contact and Follow Apps and Summaries Editor’s Guide Incidents All Incidents in List Form Flagged Incidents Submission Queue Classifications View Taxonomies 2024 - AI Incident Database Terms of use Privacy Policy Open twitter Open github Open rss Open facebook Open linkedin f5f2449 === Tweet by Sayash Kapoor (https://x.com/sayashk/status/1964016339690909847) === Tweet by Sayash Kapoor: AI as Normal Technology is often contrasted with AI 2027. Many readers have asked if AI evaluations could help settle the debate. Unfortunately, this is not straightforward. That's because the debate is not about differences in AI capability, which evaluations typically measure.… pic.twitter.com/aJrIl5lfux === https://arxiv.org/pdf/2405.15828 === %PDF-1.5 %� 98 0 obj > stream xڥZێ�6}�W���X�"�����Nl� $� �1��n�4a�������o�(Q=�,0��f�,����bG��M�y�]$ߗ��}�Zg�8 ˨�7�7�\m�K ���Sp�z?5U����(ؙ��n����W�m?_����,��a�G�"��0/�F�eZ��p���¨H7WN3~��"�s��q:�8�c}2�6N���v�7/�i�x��oiF�~�,����|�jm0�AF|��g�" uR>e���T�uo�F�� U@� ]n@ ߔ�����^� ]��M��7�T�6�O1@��RyT큇��q|�& ͒���t��� �Ğ�BP$�qiFW�`��`?Tu[�Q���� ��W��l7���nX\���lD ��ż�H�_o���_^�����aRO���wo>�XpEe���z�q����z�e�������ùx�һ ��÷���B �o�z3�$�0S�� �*1H1�a�zDq��m���O{��' ������?V�3����(�8wf��� 6��:?�&Ȁ�9��Y �x�������� ��nQ�~�Y��w�BNqo���v�q����0PB3��-���azҔ�8}dW����ߕ +� $�B�]/;U�� �#�H|g,C�# ��[���t�����E�rY00�LA��C*&�Zq� @����Gxf�R�V�Y �AI�T ޓ�am�( ��f@t����7���*�a��Jx��$K� �S ���R�����sCG!�҈ Æ���* >Ss����0LT�%��!R���a~�[ �a��cDH .N���n�~S˄ ����L(:�]z�{ ��8 [�.�0v� u �#'Ub � O;;��� [�6�Qk��� ��U@�f�1��XI+HL#}������L��)>�5�r1���ϐ�->D~���ȒYp� ��ۛ����0J9܀�h}��Pp��]b`�Kt` $�}JC}�8�?U-[s-F)�b1��#���a^���^�z��"y�ɳh�ԾXM��,/ �%����bW�y � �8��_��w�G?��%ݖqzd����l-��0� )3��h�8�g���L � � ����:@ݰ�c`v��m�����w��c�endstream endobj 126 0 obj > stream xڝɒܶ����{*�,\}'�ˉ].GS����0$f�lq���{x �p9�&��oh�{��ݏ/�������+��g%/���a��]��T0�wo�ݕL���7��A�b?ړaW� ����ҽAp����������H�G=�жUo�`�GO���d�/4�d�r�@�h��= ]e�h�͛�#�C��t�z{�w � 2����4��ԏW(��S�?����7UwB `�ͫ��y&�d���Lw֩��BCK��)SEX9��nG[��_���l�{�W�������(��Xw����K��RD���\s�ueM[��21�47�MKVJ1SИaآS,/�@�L�;S�� ��G�t�������#�r�������IT:m� ���A=�dQOwS��>Nf� �O��p��9�0Į�ٚ[K �����ޚg�M���)�>Z��>9��sگw3m�ᅈ3���m[�&dP#���l�q[�z ~�h���� 2"�6�َ�j�\0� X�t����+"��.-n������p b���>Y55#���]O�� �E/pK0��~�2����e�����;/�ڎ�?�|G��:掆sKu��N��2å@ ���ܓ�Q�l��0BA�����z/�CC@8s�� xԣ%8�� ;��HB���^�o,6\���x��O�#lP���ɴ�kY^DYRT@M�4El�� ��P\��M����ptW"`u�-�Fc�G��;�;��d���Ȁx�?�f���"��tD�N*�h2Z�:v�&����-l87�5#o�%;�0�d�)��J�yӔ�8�� �pKo9`�" &��=wa�#�]h��"}�)�9�>�&X� Z����>�/�ur���n�^��Wz���1%*���Kn+]��K��
{}
>n������������.ge�)*,��N�2-i�_���/R8yY&�[T� �b��=��� ��Ř�B����kGm[r�袬���̨k=zl��δU�2q�S[���,'M�_�� ���ݨ�5��$ņ���{�?`�j�L�fI�a+z�\x�@��d�m��5�ű�F�m�F���H����fK� �!���ц|U�Bb�;O�a� ��}�I�-3�a��N��� ͊ZA��[�#o��g_���'����z"����#�l��J���"A>8���*���W�����7!� wf�Jdar�Er�E���'(p������@`����ڈȔ��8�-��℗KK�p5�R��M^�{������R�DQ�!�T�3��&��e������e%zD��3U� 4�"���Z��)��E���%��f�z{F5�슺!K�&]��)�}�kĶ֔j�?��K螴!7i�ٵ�R��i��W��"4����o���7�HYB j��*��[�;ו ��a�_�V�:*E��"*����=�[�֗��o�]��&1��g�^��LV���sz-RΒ�X�Z �!�g > stream xڥYYs�8~����P� p)�Ah�w�w��Cᙼ[«����p����ͳ��טccZ �u���b|)���� ��g_�q6�Q��Rxݩ����l������[c/��ν}�^Vx���n�-,��g�/�?f�)D���g�"��x�1�i����Gxwk%�G����~jY�qx��s� _+�?�Ϳ��n�?ȥ� ����}�z��iM�J�4���/R�R~i^��%Z�/t��N�I�=ԅ)7�N������T�my�7���l�S�"���Z�WG� ��b�}{��v�^�O�a�ǷGI_�'Q^�(���R�J0��v���`;��4Ӷ�=�1;�_��J��U�ͳ���cUȻ�� V�5k�=����m�7�=�����f�J�nLq�I�d�Lc�%cnl�"w�ZR��n���sz� ���$�6=I=!��۽�#2{��ug2Vp�~�I\nui���r�:�Рz�[��L�X���(�5:SX�� P:0������E���GnY�G$���rK��X���WpvܬᑉĀ9|�# b�Ss��mn\��z��)>T8�՟�k�=-�go� )ҩc�ރu�p�`� �y���d�@��n��� }N���4� "uʼ �m�i)�L�H@4�qc�\�O�Wp;��h���l��dS8rF���Ǩ̫��"�%�?��;���g�����.�5�@+h��n��[�B��]���V{Nb����י:k�-f�s�`���ׁ{=l�l����Y{�X� �(�� �J�nGj�a �5�=8͎���%�BvE6�x1�52 r���`�s���F�] v�I�� [&}�*�� R�@���ψϢ� ��?���=!a=�$ w�s�e��bw��=� �ݓ��x �l7D1� c���L��-�t��]Sg��x���.)��sf���N��s��sS��'��k���ߜ����2�N榋d�X�p#4( LFu����֩�, ұ��h�j��� ������iRq�&�lZRHE;��?l�VUT�R�YZ¡����8 ��O��Q��ۥ+�O)E���45`��O�=� ��d��i� 9�w^ ������@�_�8c�� �,' �� #�۶��T�j w���mcr�gK�R(�pL�P" �E���yb���tV,|�I� �'|CA��F�(�W�7��7�m Y|������+R� V)�b��i?��É��ZcfZR7�$���t2�����s�^�r���,'q4���!����ꗶ�����9�شj��)����K�G��Ͻs��8��hZ��ީ�cV���z�9�t!.9�S�� �0�#�o/��@Ä�]%җ:Z�� ���p��X0 D�+����V��k �C^��!�@a\ K��I8�cM�+m��fwm�`!dQ�!�c�Q�M�P��/�]�&�R�x�����O���l���HoR�����.�����b6�z�2�o�����'�)�l":�~���l�� ͜�W�y ������Ý�z���P4���~���Oe��Fj�±w :4]����:��Q���`�%��s�$���0�_��,T���D$��)h8|M��K��I?k9�%BF#�u2�R�(6@i?�E�_%:��6Q���&��u�����ٛ��7]DQ�S_%����lW�d � ���SW ~�7@�лa�XO]/�:�� lt��c\6sw�? 2���>3 ?r���"a_H�# k����_�q��V�C���F��k�A0��^㥣��K������}�rÝ�RTJ'�"�If����ks ��٣)L˂RbZqϡ��p}�Rd��4��P���7�����2�gl��KQI� *q��=�?/��?G�`h��iͤ��݂w�t^� t��|�V4$�8j ��#�`!������� $7&������m��y��ґ�#�"|�-Gͺ�S!�N�p��폗���n�u����,�T��l����F>�gО��F��J�0'����� �lLendstream endobj 159 0 obj > stream xڭ�rܸ��_1/)s�44 � 7�ǻr����D���q�(Ұ�c��X�ߧ [�ۑ��+sv� 8�����'�\�ܿ���{`��K�Tϔ�Pt���n��?-�ˁ��=����U� �鸗1# �G�6Ǭ����4�g�����-��ث�?�r��Tm���/�|��t� O��q�V�S!1��h�(DL�o��qzI�3"�� �> dhC.�H���k�&��u5� {dR��;����k�Ul�q� �i�V�VUJ�C�6`C'���㖁��S��f�X+`. �Ѐ4�������̀�&S��)K�4K���g��R?K�|��p\?�)C � ,�f(b5�H��k�����=C�� I3p!u"L�+��7A`� �&�,3���eB�%՚͈��p�-e���RK�����7p�U� '�� ��gb� ��� ���ѹ���6r�f�ܝݸ+���$T�q>�m_����c���Cט�&��Тg���>�?"Qx,W���GK5�U�j���?Ö�~�$җr���i��LwR+�&W����H�AʗE�ԗ�]��ƾJ,^����Sw�@��T`�C�p��^a� ��.KcMA�kR��qq!����w})���u� 1N�e�/wR�څ��|�JT� 1h]�T�J����g�?q'��GYN�L�d_dbC\�R���,~q�I���u�T9�Q� ��r��w�Q���Q�i��`|r�S ������g ��i���I/ 6�.RE�*� p� �Ν�~=�B5��Ib2�f��I1�?Ɋ��ElH�������h=�Y��4 ����jB�L��X�L�J�m���� D2���:.�!�b?N�ߖ~�Wr� ���KV�����E�W��V��(\n��MޓR��*� 0i�'C��:��0/��ֺ8�7w�s�}���.�~ ����t%�����֬�R"�m��A�s�7�w,��?�\8�J����Q�Bn��L$ 0����% Vb>�6-��G�ۊ:�A��%'E�9^c�kv��:�STz��nѝkS���ML�s�׃�� l�$?�+�� ��Au~��=E��@��I:�n]GZ"���Fע"�06AԹ-cal���yu����8��8|���gɪzRV���U)��E *��x\�˪E��,�G\}��2z �A+�T��� �Mb��,�->L��-y����)Yj�c?]���j~5��4�͋O��] �@�/3��'�n���I��ۋ���g�(?KPO �� )�0dS���@���Tz�D �(~��dP�-� $�7��vfR�y�;���?��i��ի>/���G�/~�5��0��?^{5������6��PS$�1̵�m�����H�o�LVQ�,>.��5UF�kK�� �}\���� x���\z������ 0b3�`��g��kr������q��!r3=ЅT�X����ն���um�x�Ї�맪������k����F� �eb7%qPka�77=GT�Tq��:]x�}��� bO�l�s��-�+�O6e��F�-y��]�%(����b~�]�(��у/�_��~篔���!��Q� |%��4����endstream endobj 2 0 obj > stream x��Z[��6~���c���Kt��ȶ��&Y`w�A��5�:�4��I&�~�#���c�nF�>X�d����� f��1)$�Lj�;&��-������3m ��E�B_&J�0ÔEkт n�Bi��H�A���(��G���It���QLi� �:� ���kf�R�Y��є^Gf��{�,�G ��0�� L+���H�8@q�� �;�t��9( �-3*X�3 ��@;5c�M@f"����a��\��� ��YIL�� � �Z,�+ �~6��O�$� �M��Ĉ)���$����$FF�FBX^8;��)��Ǣq���@� ��/@ �gAH�?K7 7�Nb�A�8!�eI��CW�� Fy��P��$��Y�a�QZ�Io�l��@b�2�H:��t��Ii0�9"NxѨ{�Ɛ� RP( 4�Qw�L��A[�0PO }���Z����#E�4��8d�.�1�J�%Q��A ��B�w��h O^�dӷl���]��?���tVgE�����o'_������*���.9]j��t)�2�ˆ.��߬�7���$��$?�%��$�맃�K� ^�Vt�U�M�w��^�����?.2G m�;T�8��.�����7�g������e�{������Mى!�+����|��M�QȎ�ɺ�Y�� O��Y��W'Xv ��.�ry�;{ �����x���?vh�'S��,a�f֟4��?a�}_V=�ͤ��ߎ�ZOێ��;�����6Ԑm�'���͐�?u���a=��¬��� �s��q�8�o{��z��}w���Ig_��7}c^�=�J��%S���!%ӗ:�쉊�4�|���4���w���~v�c*�^�n���O�f3��;���� fU=ɑ 4�Pvw����af?�J��b%zD+�5'Ү��7}�y��]�����u,ݜ������_�{$ł�a��Y}E {�r2}�p���/�"�L�/�:�� Uu�LߤU�)g)������ ��� �h�B��ʊ�x5�PH��_�yr�\��Q�5Mh��4N��l�6�m �j�Cې�L�n����Y~;�~W��lpɫ�ӟ���T�h!���W*pC�kT ��S�b̢3�I�i#�r��1B $�����U���.-�җ�b�z��� ��B r@��y8 ��g���U�1�����p�E�h�y "�$ų�1p����R�����X�fm��4��aL����t����x��#�,��r�&?1���\#%�2K�9�a��f�̲F�����|^$FL"��0��"����Պ�l�$`1r�d��2d� !�t�r0Hg�iP�� g�H��xu�� S��|�t ��* ��g��&b�������%�����Rw?�E����^�8@�j�I��I[�H��Q��3@Ԙ^�B�QZC�4�VK����_%��= 9�PI�5 C���(����b�H��ur�U� �5�1�E�\����E M���[R��髗/� ����������D���u}W}3�&w߱��-�UR�\���bVM�I�T)�/� d����c�EiC���?x����5��i�Z�y�M+��x ΓN-\Rg����� ��~T�a��_� _��{=jM��A�J�(�4( �!)��t���PQ������!��:Dy�� > stream x�uUKo�0 ��W�2�&U/��nݣCv�[��k+�0������'�Jb�. �IS�ȏ '[��������D�s�2I�B0��I�_�^0-�Ƀ��L���n��s����nC�sl: �q��{P�� ^���$+�!�ə�3���c�J�����)�Oć�*#�O��݂-�]��~��z�Ȓ}��=����n�i���+m�:�7}���� O��R�?��^���a��X�� U�F ��)��m��� I�e�+S�%ϩ ���C:�V�FE�5���n� Q�.�Y�r �Z�� ��!�LA�m ]h*&H�aZ�n Y�w,8xu�!M��� ��gI|�⭱��N* �kk���S���ܬ��Z��%� � e��s[Ü�%�ퟭ�d���/[�P��%B*f���K ���u=����y]�V IG1g�V�� Q�l,- +3Ӎ�q���������I�W����u>c���Tq�:hc���LV������8�9�a,�0@�g(�鈶zc��zD�j�7���U��u�y���)�.h�V1�~G|��f:[b��7�MT�endstream endobj 155 0 obj > /A2 > >> /Font > /Pattern > /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ] /Shading > /XObject > >> >> stream x�řO�]� ���S�M{�k��(�;;M�f�4EЅ�L�� 'A���#5�s23:^���=�:"�;��t�������ۗ���>��� ۏ�{���G������o�������]�lU�t��������s��̊���v��.�+s�wWOMe�����)�|�n ���O��߷۳�[I�� �~��:���������,2� o������VTG������������VV�v�v�1��v��v^@鯇��_�/ V�ʹ6��XW�>�|����Ë�ó?��?$���۞�����]|}����M:�'I��.�UnNw=����&J��Z��ߓ�Y�)m}W��������0u���L��}�p��K�/S�SU�n����;���yKw3U ���@�y/���Ǐ�Vl/͛���Aa꼥3L9JA����L=~L5��Y)��绅��YKw3E$�~U�/Q��!��v�o3�yM�윩zM�V���������yW�����A�鼥��2��֭8��L�I ��ѻI��EҤ��?ޒt =��cKW$�X:C�#�ݪ���ΰ�BF�sj��+}'���6��e��ÛWo.�_~�e��Sh�WK�C�B�1��5�e��z-�}�L�O��͊É#q ֦��i�GWQ���(Q7l��u�", X���e����jabh��-8e�ʵ���P��o�e$�8vW"fR��_�E#��6�6u/պ����ҽ�G���9'�Yw%@�;�4� ��3�:d�EH0��Z#4Z��2+1���)%2�ʰ��J��N���'G�,)_��X횸Ydo�L:T�L �T�8�� R�[8�vH��K�8p�s�'c�"�#�a����Q�4 �k4���٘ �o���f����%Ka���B��#qGj�����c_�`�Π�Q�$��3��ZȕPԬ-������)Wz�̢�Q� ]���9�S]}|�� R�B�bې��ǝ�C�B��d.%�M�jь�ƺ�ŷ[�E���{�¬cѦ��q�!�D�����8�=n���f� ���僆'q��(b� �(�6� �Nbx��k�2ƞ�f�N� &���L�N�q4]1�N��\����tDbE@����wW���E�q�su�-DJ�hڱ-� �B�)K(��z����Z� S�'�=hqKĖd���%��QD�e��1V�}�I�!��F��M;��^���8"ڄ�V��D��@Bo=��R�ڠ��k$ü.�&�t �ݬ%�$j�4ʅ����6UB��ט�LvY���y5b�D�xFI�oт��1A���za� 5�>)��nJ;��EkB�kb%% � 콧rŋ�ޙ����fk5��kC#"�{Y��,�jD��ꪇ3B���������GOV-ޫ��2�-:2�)>���J�5:�u�H>��+ > stream x�5�� �0D{���8������m�-�=��l`d��#����pSLRN�w j�7;�%��4�z.�q����Q$��D�˕g�_|> obendstream endobj 188 0 obj > stream x�5QIn 1���@��'�����Q/��$��F$>Đ�(W|��4��;Y���J�佈 �*� i�rV3 ���3�Yf>�;PM�qv�4�A�i\G��0� qc�Y�bH��R����*�G��,�N��Jݢg�I\v^Omi_u7�;��"�,.:�n��)���!8-���T��&~f$xʞN�4��A���{AGn&d*�PF*��4ƌ���h&��������� CMT�endstream endobj 189 0 obj > stream x�Mͻ � О) Q�"ٿ� ���t'�`� nqX �;���S�}��Ps�DX3%ڄ�_��0ØXȶ�7W�r���endstream endobj 190 0 obj > stream x�3�4U0P�� ��� �F� )�\@>����� �4XiLEWW �� Vendstream endobj 191 0 obj > stream x�=�K� C��BG ������[˙t�c�/DWt앇�Ɣ��hcO�-\�tNS���f]��a2�o����Ta�������f��U31�7�`*wr���8�ے�lWaН��0̑�=Ss�s���#_rkaЙ�-��eL��a�l- �Aw���z��0���A>dendstream endobj 192 0 obj > stream x�334T0P�5f�& �F� )�\@>������L̀,cSS$��625���pDW Rk�endstream endobj 193 0 obj > stream x�334T0P�f�� �F� )�\@>���eh`f�p�)��Y��P1��������9�e ��L��4DjhWW ?��endstream endobj 194 0 obj > stream x�=�Kn1 C�>�.��ٞ�(����'%��Ej��LYS�4����p���;l�ff�Z������b�|��F]Y'��f:��Q96M���,��.x�&�[�?�Р�5. 7tW�e)4c���{��2�jL]lR�{ > stream x�5QIn�0 ���� ��'Š����R�@%��%bc#/1�9����5�&~grW�,O�G�I���^v���e?3��VΤƓ �E6= j�D�(g����(���g�U!h]ݧmc�FJ�ӿ MM��D g�4��Ͻ���N!Tendstream endobj 196 0 obj > stream x�366W0P04�FF �@V�!H���D!� $b�Y@�8�&�+�i��� �-��*,�lW ���endstream endobj 197 0 obj > stream x�=P;�D! �9�/�$�#pF�-f�߮�)PL~�3$��G 1���%������B�n��� �CR �z�t�6�:�3?a7c��E1��t�=&9�� �se�VH'��"��3�)�*{�x,��6['�=� �RR�ɥ���?mʔ :f�,��dM8˻IR��2��v"}� > stream x�=RKn�@��\����yRUݼ�okCR�*��1�0}ʐ��K]q�ɷ^�[ �ܗ�!�]t.�8G� 2�*D�ͪѡ�B���N� �}9��/��װ ��=2A�$�)B�nQ�Aa���P�Y��Q���2jo�� c� �BmH��@ D�T���g$��gb`�Ѳ�TD�{�Lj�Ψ��D�>OM�(�L-V�nS_����|t*�4���U�� X�y�9�H���l!�:n���3�2�`K9`���G��Yu����t���pL��~��O�t�Z�u�r�@�MA��F���2>��)z�,���F3�a�����r�4�k"�X"��bD��ls=��L��9���l�֡�33*!�ں�j�@v���p��?3�mendstream endobj 199 0 obj > stream x�MQI�0 ���@!^���C�C�����9 �����X 1�,=��!s7�~�ٻYz������"SQ�R�.bB]�ϡ�=�kY���9,��s���3�c}I2���!N uZ�¸kb��Z� ���v�Bz�M;"� �2;-+�{��n��?�R��������\�endstream endobj 200 0 obj > stream x�E��� �TA �O&�����2|`�� �yF�&^XJc�P�@��r-��p'��T���wi�IU)ܤy�g&��y�c�� c�# > stream x�ERKnD1ۿSp�Hᗐ��[�L��� `lx�S�dʰ�%#$הO}"C���4r$"��I v���kK��쑨�����F��Z��ѥ͢Q���9�����{���z\H���!��P�,e�r %A[�w����+�a�k.��e2����\�}�b�.���� ������6)h U��I£�znv�l�U�� 30r��%[��� �}]�E���+@QY@���v��1�Hp���p > stream x�E��q1 C�%0� ����������AzB bO°\WqatÏ�-3 G���2c��X '� ���Ѥ�v�����b���\/:"̒�@#|:�Ǔ��3�t^�!��**�na.��@�R�ԏ�Qꚡ�*+kj�W�]J�>.�2NݽZὕ�=��?v:�endstream endobj 203 0 obj > stream x�32�P0P�4�& �f )�\�V.L,�іp "��� �g 'endstream endobj 204 0 obj > stream x�E�Kr D���#��� > stream x�E�;! C{N�#�#>�f2)6�oc�lR��X ����ڂ�肇�%��gc��6n5�u�V��h�R}��i�t�h6s+ �fz���:����r����Cp�_��b�9�������S��6;�������ܬ�~+�UaiYK� ���f@ �S9ګ=? :�endstream endobj 206 0 obj > stream x�5Q�m�0 �50 ~%�� H���;کHӼU^2�\.u�*Ya�Cu��|�f������!�S֖{��yJ��,�p 6� �s ���+�Q�D�n���Y@�蚫�Xb �����V�)X���v$� r1Y�SE�)��|,ّZ��rY �n}a�8쳋D�y�L$�d�l�>[���E6v�8�Ѐ����$AS8�>�a�j���('89֯�1� N�KR�W��ϠA��3 ���.�I�wcJ��d)�6 ��S��b��Vo�����fQl������ܠbod�[���7�}�_{dendstream endobj 207 0 obj > stream x�=P��1 �]�X�z��g�K���H��"�ER*5��)/u�%YS�:|��y��nd6%*E/��%�� �}����Ֆ�C4�h9~ 3*��K6�p*����3� mtV��[ Ф` r� ���" JM��r��^�>��C���5�X��s �S�'�no��^�����#]X�.i5AM��Z-���^��#� @���q�R���kp��/�'S�endstream endobj 208 0 obj > stream x�E�� �0D{�`~&�>Q�� %n�'�=\ �)n3�C� ,�]q���0 \GP���aվ����R�������Bfendstream endobj 209 0 obj > stream x�5RK�#1��)��T��1>O�^�E��ۑ�{��$f�!C�?�R{� ���,[���ɽ�J�%s�Քϓb������"�BG5�$ L�(a�ϳ`���bvm?Յv�����x2@f)��K��P���$m��|��3T9�Z��b��h�!7��م�>��IdȠbJ�� T�ك�����;�9�����Y�ֱ��,�A�q �U\�U;v�F����j_ov�R���y8[f�=�����1�p�@�.z�w�CD��j �����y�������B��9 ����x%�a䦴��� ���Y�E����4`Wdi���h��4���U����endstream endobj 210 0 obj > stream x�E�91s��'H\���˵�����G��4�M�zh>�F���k�,J�D��P�c�;(HLHc�H��#�X�]��T*�R�i�¿�Z��D�p����T�O�\��M���1�T�;��I�^E�Z1��D0�� oE&����پ�`��� �8�� L�7�endstream endobj 211 0 obj > stream x�5RKn1��)�@��O���ok�L0`�)/Y�K��%ۥ��[�%�Bޏ��R����-�G��- Q=X�/"����:�pJ��W���I �j��3 P�@ƻb2R�$�siq�r&�����'����=Y��ߓK��MN����I�]I����JD���}�B���<>G1ɀ�6�m�&��%��:���f��U�`��)�W ��l�[�����0��0Sb����be����5��Q�]�p��w�h���c��G�� i8����^8�^�9��c��, ch&�z���О��a��Z��T�|�endstream endobj 212 0 obj > stream x�36�P0��C�4 �Rendstream endobj 213 0 obj > stream x�E�K!D�������q2�s�퀝nc=�T����=.k�!��EY>TzRV������!ܠ,��I*�!P���-en���6 ���S�����0���l_�["u�?��*/��ȥֻޢ���1 endstream endobj 214 0 obj > stream x�5R9n1 �� } �n��� H��� ��qtR�;ZTvˇ�T��t����7�we�� ɫ�R�H��V!��2��5g�R����qɂ?å� ��������X#�D�2������f[~�i��x��+�-�XR�8���y-������ V�~A�G�}�TX ��Ip��P��v��H 9{,vQ�3(1��E�A�%��9źt���б��،7�:*�[ZYc�>��,2�zX�t��ǃ 4���� 4e�����9���QVx�C� �1�!��� J� > stream x�-QIrA��+�f��c�+����� �@tZ⠌� ,W�呫h��31���~+�mX�@T IcP5�v�z��q� ְ̓ f���)����1g��e��]�=�Aɬ\�%L�a#g?�2.�s=7 �FX�a�nV3�4�H!�����A��F�6r��/�t�U��1�s-m_�M8k��X���]q.'/���6Qi��@H��n�s���xK����{����T`ś�n@6�����������_`[�endstream endobj 216 0 obj > stream x�5Q9! ���@$�/xO�(���͌�V�\���!��R��!�S>z�.�����j4疴-s����/����f���i��� ��n�c����!� b(,AͩEm�'�5�^�ޗ���S�Ɩ�V8����=O&�t2ԫ5� E" > /A2 > >> /Font > /Pattern > /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ] /Shading > /XObject > >> >> stream x�͝[�9v���W�� �����c�ذ�t��1��JC%�%������f��f�D�R�� U-1�� 7�"���w���=�y������wy������;��}7��?��Ov��}������`��s�������nMLs 1�+������ÿ?|�G������!������m���.��9�t�{�n�+s .� �Θ.��t��y������&������y�Su�N��N�g�0}�G�L�5��-��o����4�?�'�1Uck2�xl�Ӑ���BK��ǜ?�����_������=���pS�-!��侇���x��z�7�O�����{O?�F���%���)��K�����ɔ��$Y ��InR�6��Q���B�OA����W8�S]�Ŋ�1�9�V�T� Z~��� �������\i�[g��F���vP�V�Wf����#� g��`Kƕ�۽4t��i,��\��WtF�RZeEsB߄t둵lB����\f˱fМڪ4K��F�D̠�V_j�.���!L�M4��r����赩��I�gy���h*hFړC/м�T!x �n��!���BM���t�|a{'xQhUsI���AQ�_>錭���OѲ��Sr�p��n�ݧ�7������O���������/o�~x���� �Y��w�e���\�&\�X�hГ�LK��G�(Cg� ���;>�7T�a0�v(lO>.�o� o\���a>I?�� [I��m.������T���G�l�R���� ii(�p��'��q�_�(A��z ��6�7Iz�1����p�֊6�"�Z�C�"�!�%����T�����%=ؙ��Tc:~#�3@U�>�*�r8D��&24e[rD����FE[ၭFP4ϩ�����.6ю� �"�22L'�T��f=�L�8�W��"�("��6I��+�w �x��o��ef9�t��h�ԱP��/^P��7��-=��{�:G�HT�R�Z�B8�B�ʺ(�`��2��51ΠULn�O8";?}٫'Ѽ�4o�M3'�a��6�\��#�;����p,CK�1���R����i��s��܁�(^��C���ӏ�c��;u�j��w��N��J��K���oܦ�]� x� �P�� ���$��^|H��ι3�Bݹtӓ�{WY���o� 2�(��++���v���!c����MP�e�2�� ���h��Z ���Ȑ�C`Qk���SF^T�l��S�TD����'�`"T��0�>&41��٘��O.QY����6pq��F�=Y�Ɲ+��,]e������b�ZC� a�!+s�!R*2�'fh�B��P�Q���^VB!ol�ܠB ��=��br�3t� 0��0���o��(���b���4M���QlJ0([ ѥ�^ٜ8^���H:�ecD#�V���e18e�C���[P��%���&[iDB�@ $Yf�'�U�xW٠��&���A�y.�2=�M�ߐt�D)Z�\!ݡm���}dC��G3h���(�l*� �Nd4�� )W52U��X�� �2��� ��1��OC#���g����v.1�-I��ϐ˔{����s� �M:O��`:#��\v!Ux-j �^��CPE�A�gYP͐˶:΄� �Z�J|�hD���V��t�E��G��m����7s��P��?~|�����*���ܘ�f �J��(�!����n邒 @�'��i��r{)pk �ǐ��"s�vr�c�Mh+�(3�o4�����i�+Wb0pr8��!4�?� 2��?�+k��G���ژF����rurPc��NT65�A�7mc�"��m[���M�W�f������j�6l@]p���QkI:�h�/���*]�t�� �蓨��\���!�~b^��ƌt��b"�f ����R1������o*τ`��f�?�pn��T����Q��$T��Qh�1�ܠv�R*����%D�/:'0��6.���8ђ��P�E�4�\���̲ka��iQP�9Eٗ� 架�i�E'~b:�5I�40���� �_� DkX�-������ی�Dn�x����H���� *�d�춢�(��,�ߏl�����o?����t�e��&�oS�� ��Å���Y�o��#�w�K>_JYNL0r��c}�Rj��sR�.|� ��c�m3���=f��'�9s�)�D��㫡.�Uܗ���O.�B\/��\C)�;ݹ�cPE�w�w���b�� ������ ���E���v���\3��;�1� j|sЎɷ����B_��O��q���;�� -��3 � W����j)��L�/���S��|ȐY4���� ��P�����˦��B�͙ �Yd����XǨY���%3(ҵ(G�7��a� ���%=r���d�ݒ,�B�f���R �7��m�a�ׄ�c�,�=!�d�( �[m��v�w˳5b�v�d��ז���ߕce ���V�]�M*�o87S S�Y4�4/0Z��E z �U�{r�1ϖ|*e���Cl��+���6E���2�3Ye84�����܁�ڥ�wZ�k�x����ȣ�e$���������O'�4{�;�^�k7(3g'V�5��^?�'ս7 �S�����>f��v�m�w� �pc�00�M�W����b�F��!���A���2������o�!�ƭU�a�6�G��� %�U A�����-%�b��� ieG���������>�V�k�`1f6�>��;# ����� a��h@����^���b:rf=�), �ʂp98wX���S,��^1�-,(%9��6,� ���-S�"5�Y^��aAN��6]Q� ��-�U� ��NY�� ra�Y�b�[+,�tµ�b 2�M�-,Ȩ H�%(Z��� j��hAF{���H r�h�_b�-� ����N 2��d��qA�HY��o���b� �\X��78� _�҂�0Ds���N >S�_V���ܧ�w�����~8\P�3��zTl���������G��{��d����g�>������S6Yz �N�@�&#�*���CI5~!�d�3��佳��k8C�ծx����a�>��_��W����~�Lvk�x��,���L�ro����'�2p2 ���ҏ:w0ٝ;Z�����R��.���r.D���ao=��k�A�Ym�c9bhʘk�\ �S� �g�#8_�¯����+�K>h��C[Xsm��D2��L[�>� �6�3J:A��x:B�|d�$�{T+A�c[�.�S|�$6�L1��ϯ�MA� $�)�3'� ��6 &ߦɶ!dD:����!�ZY#;B�g�5-b[!���ip"�Q�.�/\R���ۑ%�͎z��0���v���l�R��J�� )���n����0��P�BDiw��f��\�BI���v����� +��a�� Y�9�M�Ѡ��� ��BNV8M���l]i�+C��� D��:C��y��!d�2�x�&�b=:�tHЕb� d~�NX��z���zb W��@�ǃ�C����+�0=`��ӏ��c9sA��.)�cn���HW���+����'���!�*`}ȵd��ׁ�gg ��5��΅ EU��Q�{��9�{�� Řq���m�w� ��@��\��@O? � 6�6�|��\���5� rt��|�� ��"��e�s�e2}�o�eױ�g�䆽y��L#C5�wT?��yN?݁7a1ָ���zc��;��n��!��T��{�a�m�������Ƅ 7'���C��A�c:��~_� �1����ݫ��w���;ڼ�^�0(����i� �{�*�eG �G�P����1�r`��t"LVHm��� n�b��QKn�b�+��s�7\� dL���_H,�D ��˕���Yn�\�N+��o,�PQ�oL�B�l&+�M���ҵ3�V���sQӴ3�V��� �a� �.���3z����k^����c���� ��h�MP��a��z�.ѯE&�`ܖ���ǡ���b��7�,�� �o\:��'4`N�op� �6>��o���%�N[���Mn@�����o�h!*��C�o|DZ�P�>e��o|r��wM#�F Ud�i��~��PSO��J�bqx �r��J��1h�����o��ౌ�=�+���yn�S�o�[j���.t�������k7��>��� \�M�\]`�E�V�/!��l��8f/�²�����ׯc�%� ��U��J�v��)�r�y�t� ^�dɍ��Ys���lmܕ{�P�\ݫ��w���;ڼ�^.�Z�D�Ku�$�:�y1�aWOi W��J�^�[���AɄa��j�R�9��Vec C���Z傾 %� �Jś��D�)j�92R^N��Ԫl 8��u�Vy?T���[�}a_K]�U��EY�Vت�����*�T~*��*3�h��u(π�i�V�l���i�V���̞lŪ�L�����6tCa�L��K�Ӏ�2�x\� ��V�g����wl��7��ig�vl��v`r [��"� ���+#ѦZ���}�oU+nUrS�r��ʭJU�ل ��t�`�]Pܪ����6��*�����l^(n��(tFLn�� ���9/�G���/E!邪�� �ɗS����#v������ׄ&��6�ӏGy�0�r�Q �$s��Cx A��czZ� _@��z=�.�F�1~m�u�(�:���h����V��l�r���厝�=}.X�q��pԹ������ �:K��� 3��'��K��̅?�m� , �PG��v ��4$�7�� ��D��h�x''��V��vW� ��elȜ ���m�50�9�R6�bj'�(�W"E�PI��/�_J�� �+q1�6�+گ�7_RZJ@�`�}0� ���wU�+{�d���e���NeR�/�%f�Nʫ�_��l0��/�� �Y7_��A���_�tp� l��ڲk�;��rA�� �K3���b���b�Go�{bY�e9�F��-�݀���_�=��Ϋ\�_�>rMG+��h��:�˝Zx|��N�R����lN�� �&%�K � �ʜ����;��H$ �,H�J�>S������ �;�_O�r�8�[���Ǐ�� D�B� �֗ ��E�&~9���������O���~y�v�zW�W vڗ=��-li_)I�������g����vޗ ���p_�[�l����4���)��ڈiZc��h%����ώ�J|?r~��A�ʗxa{9s�}�#j{F�}�)�/��K*vC���{ ��ѣF�{@�������|��S� �ì�qz�_�� t6 Z :$���^���Mb�.?Ll� g��Qwt6�����x��g�XyX�.�N���v�� l?��f �>n�5����X6�0Sx�یcU�.��r�| > stream x�5�� �0D{���8������m�-�=��l`d��#����pSLRN�w j�7;�%��4�z.�q����Q$��D�˕g�_|> obendstream endobj 223 0 obj > stream x�=R��1 ˷ ��_R=��q`���=;11K A�nQ9?m%ˏ��egI���m���� s�q�b��$-e&�g���q j��3�!I�rR�"�������_7��tlPzK���Tþr�;��]�샦���3Ӯ쀥�ԤN��#,¤�HE!����,��۹~�]�XO� > stream x�5QIn 1���@��'�����Q/��$��F$>Đ�(W|��4��;Y���J�佈 �*� i�rV3 ���3�Yf>�;PM�qv�4�A�i\G��0� qc�Y�bH��R����*�G��,�N��Jݢg�I\v^Omi_u7�;��"�,.:�n��)���!8-���T��&~f$xʞN�4��A���{AGn&d*�PF*��4ƌ���h&��������� CMT�endstream endobj 225 0 obj > stream x�Mͻ � О) Q�"ٿ� ���t'�`� nqX �;���S�}��Ps�DX3%ڄ�_��0ØXȶ�7W�r���endstream endobj 226 0 obj > stream x�357U0P�� ��� �F� )�\@>������L̀,CKd���!�eba��26���"X@lM��� �4 5endstream endobj 227 0 obj > stream x�MQIn�0 ���� ���yO�A���%��#K\���D^�P�B���F^ ���֜���?�F��?T[ 1Q$tQ7�H7� �~��W��X�w+�[:v������*�B 7!Di�^�����um4��6�'��G���I���)f�l��m*V2 7����TFZ�6�2���2ZOv�&���'��q�.;;b ���>��|����i���qA"4ť�g���x� �O\&endstream endobj 228 0 obj > stream x�375R0P�� f�& �F� )�\@>���ehif�Y&�@���) " � a��`���9� $�6f[WW ֔ endstream endobj 229 0 obj > stream x�3�4U0P�� ��� �F� )�\@>����� �4XiLEWW �� Vendstream endobj 230 0 obj > stream x�355W0P�� ��F �F� )�\@>���ehif�Y�@H�a ���s`zr�2�� ��endstream endobj 231 0 obj > stream x�=�� �0C�L��S�TU��׆|z�[ȸ �څ�o��'u`]^Bd��;�Jf��&��$q�D�;MJ����������endstream endobj 232 0 obj > stream x�=�K� C��BG ������[˙t�c�/DWt앇�Ɣ��hcO�-\�tNS���f]��a2�o����Ta�������f��U31�7�`*wr���8�ے�lWaН��0̑�=Ss�s���#_rkaЙ�-��eL��a�l- �Aw���z��0���A>dendstream endobj 233 0 obj > stream x�5R;қA�S��Y��y�ɤ�s�6;,�B��x�!�Q��%O0^'�w� > stream x�334T0P�f�� �F� )�\@>���eh`f�p�)��Y��P1��������9�e ��L��4DjhWW ?��endstream endobj 235 0 obj > stream x�=�Kn1 C�>�.��ٞ�(����'%��Ej��LYS�4����p���;l�ff�Z������b�|��F]Y'��f:��Q96M���,��.x�&�[�?�Р�5. 7tW�e)4c���{��2�jL]lR�{ > stream x�5QIn�0 ���� ��'Š����R�@%��%bc#/1�9����5�&~grW�,O�G�I���^v���e?3��VΤƓ �E6= j�D�(g����(���g�U!h]ݧmc�FJ�ӿ MM��D g�4��Ͻ���N!Tendstream endobj 237 0 obj > stream x�5O9�! �y�>0U�@����6�����N��!���x�##�f��Zd f�SLſ�����"�� a��p֬�n���v��X�6��Y^��L�Wg.�ci��9�n�]��u��SXG0� �t�Ô sT��Ɏ2��8�'�����,v9~� 6�!��*z�6���y�rA�]��E��% �Qb��_�vt�(sB�A.!��*��P����RQp�>�謟�_\-endstream endobj 238 0 obj > stream x�=P;�D! �9�/�$�#pF�-f�߮�)PL~�3$��G 1���%������B�n��� �CR �z�t�6�:�3?a7c��E1��t�=&9�� �se�VH'��"��3�)�*{�x,��6['�=� �RR�ɥ���?mʔ :f�,��dM8˻IR��2��v"}� > stream x�=RKn�@��\����yRUݼ�okCR�*��1�0}ʐ��K]q�ɷ^�[ �ܗ�!�]t.�8G� 2�*D�ͪѡ�B���N� �}9��/��װ ��=2A�$�)B�nQ�Aa���P�Y��Q���2jo�� c� �BmH��@ D�T���g$��gb`�Ѳ�TD�{�Lj�Ψ��D�>OM�(�L-V�nS_����|t*�4���U�� X�y�9�H���l!�:n���3�2�`K9`���G��Yu����t���pL��~��O�t�Z�u�r�@�MA��F���2>��)z�,���F3�a�����r�4�k"�X"��bD��ls=��L��9���l�֡�33*!�ں�j�@v���p��?3�mendstream endobj 240 0 obj > stream x�MQI�0 ���@!^���C�C�����9 �����X 1�,=��!s7�~�ٻYz������"SQ�R�.bB]�ϡ�=�kY���9,��s���3�c}I2���!N uZ�¸kb��Z� ���v�Bz�M;"� �2;-+�{��n��?�R��������\�endstream endobj 241 0 obj > stream x�E��� �TA �O&�����2|`�� �yF�&^XJc�P�@��r-��p'��T���wi�IU)ܤy�g&��y�c�� c�# > stream x�ERKnD1ۿSp�Hᗐ��[�L��� `lx�S�dʰ�%#$הO}"C���4r$"��I v���kK��쑨�����F��Z��ѥ͢Q���9�����{���z\H���!��P�,e�r %A[�w����+�a�k.��e2����\�}�b�.���� ������6)h U��I£�znv�l�U�� 30r��%[��� �}]�E���+@QY@���v��1�Hp���p > stream x�E��q1 C�%0� ����������AzB bO°\WqatÏ�-3 G���2c��X '� ���Ѥ�v�����b���\/:"̒�@#|:�Ǔ��3�t^�!��**�na.��@�R�ԏ�Qꚡ�*+kj�W�]J�>.�2NݽZὕ�=��?v:�endstream endobj 244 0 obj > stream x�32�P0P�4�& �f )�\@����B.H ��� ��%���g�� m� D���D����J %��endstream endobj 245 0 obj > stream x�32�P0P�4�& �f )�\�V.L,�іp "��� �g 'endstream endobj 246 0 obj > stream x�E�Kr D���#��� > stream x�E�;! C{N�#�#>�f2)6�oc�lR��X ����ڂ�肇�%��gc��6n5�u�V��h�R}��i�t�h6s+ �fz���:����r����Cp�_��b�9�������S��6;�������ܬ�~+�UaiYK� ���f@ �S9ګ=? :�endstream endobj 248 0 obj > stream x�5Q�m�0 �50 ~%�� H���;کHӼU^2�\.u�*Ya�Cu��|�f������!�S֖{��yJ��,�p 6� �s ���+�Q�D�n���Y@�蚫�Xb �����V�)X���v$� r1Y�SE�)��|,ّZ��rY �n}a�8쳋D�y�L$�d�l�>[���E6v�8�Ѐ����$AS8�>�a�j���('89֯�1� N�KR�W��ϠA��3 ���.�I�wcJ��d)�6 ��S��b��Vo�����fQl������ܠbod�[���7�}�_{dendstream endobj 249 0 obj > stream x�=P��1 �]�X�z��g�K���H��"�ER*5��)/u�%YS�:|��y��nd6%*E/��%�� �}����Ֆ�C4�h9~ 3*��K6�p*����3� mtV��[ Ф` r� ���" JM��r��^�>��C���5�X��s �S�'�no��^�����#]X�.i5AM��Z-���^��#� @���q�R���kp��/�'S�endstream endobj 250 0 obj > stream x�E�� �0D{�`~&�>Q�� %n�'�=\ �)n3�C� ,�]q���0 \GP���aվ����R�������Bfendstream endobj 251 0 obj > stream x�MP�m1 �� 50���� > stream x�5RK�#1��)��T��1>O�^�E��ۑ�{��$f�!C�?�R{� ���,[���ɽ�J�%s�Քϓb������"�BG5�$ L�(a�ϳ`���bvm?Յv�����x2@f)��K��P���$m��|��3T9�Z��b��h�!7��م�>��IdȠbJ�� T�ك�����;�9�����Y�ֱ��,�A�q �U\�U;v�F����j_ov�R���y8[f�=�����1�p�@�.z�w�CD��j �����y�������B��9 ����x%�a䦴��� ���Y�E����4`Wdi���h��4���U����endstream endobj 253 0 obj > stream x�36�P0P040�F�@���B�!H ��� �Y@�8�&�+�+ � �endstream endobj 254 0 obj > stream x�E�91s��'H\���˵�����G��4�M�zh>�F���k�,J�D��P�c�;(HLHc�H��#�X�]��T*�R�i�¿�Z��D�p����T�O�\��M���1�T�;��I�^E�Z1��D0�� oE&����پ�`��� �8�� L�7�endstream endobj 255 0 obj > stream x�-RKr� �s ]�3�� �s--|FЀ_䟯����i��mӮz�L;���lOs^&_Zg�:�����R���3�s�(�VF�v6Hj\lo��XJЅn\F�#���" > stream x�336S0P�0��� �F� )�\@>������,́,# ��.C c0mbl�`fbdY 1 �2�� ��endstream endobj 257 0 obj > stream x�5RKn1��)�@��O���ok�L0`�)/Y�K��%ۥ��[�%�Bޏ��R����-�G��- Q=X�/"����:�pJ��W���I �j��3 P�@ƻb2R�$�siq�r&�����'����=Y��ߓK��MN����I�]I����JD���}�B���<>G1ɀ�6�m�&��%��:���f��U�`��)�W ��l�[�����0��0Sb����be����5��Q�]�p��w�h���c��G�� i8����^8�^�9��c��, ch&�z���О��a��Z��T�|�endstream endobj 258 0 obj > stream x�36�P0��C�4 �Rendstream endobj 259 0 obj > stream x�E�K!D�������q2�s�퀝nc=�T����=.k�!��EY>TzRV������!ܠ,��I*�!P���-en���6 ���S�����0���l_�["u�?��*/��ȥֻޢ���1 endstream endobj 260 0 obj > stream x�5R9n1 �� } �n��� H��� ��qtR�;ZTvˇ�T��t����7�we�� ɫ�R�H��V!��2��5g�R����qɂ?å� ��������X#�D�2������f[~�i��x��+�-�XR�8���y-������ V�~A�G�}�TX ��Ip��P��v��H 9{,vQ�3(1��E�A�%��9źt���б��،7�:*�[ZYc�>��,2�zX�t��ǃ 4���� 4e�����9���QVx�C� �1�!��� J� > stream x�-QIrA��+�f��c�+����� �@tZ⠌� ,W�呫h��31���~+�mX�@T IcP5�v�z��q� ְ̓ f���)����1g��e��]�=�Aɬ\�%L�a#g?�2.�s=7 �FX�a�nV3�4�H!�����A��F�6r��/�t�U��1�s-m_�M8k��X���]q.'/���6Qi��@H��n�s���xK����{����T`ś�n@6�����������_`[�endstream endobj 262 0 obj > stream x�M�IC! C�����3�� �1Aݢ-X�v���s�21�F�� > stream x�3�4R0P06 �fF �&� )�\@>���ehd f�pY�)XX &f�P!��.cSs�@EƦ`�?�+�+ ���endstream endobj 264 0 obj > stream x�=���0C�� �@��)�|O�j����#K� zc������Æ`����%Tk��@%7ș`���G�zb8\�������f}��B�%h�{�Siܦq�5�)ꜣ����g�4��4��s� �{�S��������w.�endstream endobj 265 0 obj > stream x�5Q9! ���@$�/xO�(���͌�V�\���!��R��!�S>z�.�����j4疴-s����/����f���i��� ��n�c����!� b(,AͩEm�'�5�^�ޗ���S�Ɩ�V8����=O&�t2ԫ5� E" > stream xڕْ۸�}�Bo��2\���T*�v��V���M��őS�L��ߧ � 3k��@���P�9n��ۛ�|�zw���mX��i�6w���HJ�� � 9l�o�N���nE!���B���?N4���#��� Pٳ��݇�mnY������9"���k�M��H8�� ��ѷ�h��k`j�i>j"�������xfCx�ۦ��dw�gl��.SۆV�A���� �J��#h�0S��/ �S3�nA��; �x���]����> ]5�D��" �k� �v�G�z�a�4�h �ohj2�ى� �����# �e�w����O�j=�D���#�BE�0G�GF��O��R����0�\���}un�f�8�F�u�!���6ڊ��5��4��GtHz�̶oНu )�ir� �J�mk����;8�#B9�L�pI�r����b�7���y*'