hackcv
#tw-jasonwei · 19 条相关内容
行业资讯精选 83

In addition to agents and coding, Muse Spark 1.1 is also really strong at answering health questions, a steadily growing use case for AI. On HealthBench-Pro, Muse Spark 1.1 achieves +5% better performance than Muse Spark 1.0 and beats all competitor models except Fable/Mythos. Excited for more to come· Muse Spark 1.1 在健康问答领域的进展

<p>In addition to agents and coding, Muse Spark 1.1 is also really strong at answering hea…

#tw-jasonwei
📎 Jason Wei🕒 07-10 01:05🔗 rss.xcancel.com
行业资讯精选 82

RT by @_jasonwei: Excited to share Muse Spark 1.1 with you! It’s our first model available in the API, built to excel in agentic and coding. I’m incredibly proud of what the team has achieved in such a short time, from 1.0 to 1.1. The momentum is high, and we’re still training larger models!· Muse Spark 1.1 发布

<p>Excited to share Muse Spark 1.1 with you! It’s our first model available in the API, bu…

#tw-jasonwei
📎 Jason Wei🕒 07-09 23:30🔗 rss.xcancel.com
行业资讯精选 85

RT by @_jasonwei: Excited to share what we’ve been building at Meta Superintelligence Labs! Today we’re launching Muse Spark 1.1, our strongest model yet for complex agentic workflows — delivering massive gains in agents, computer use, coding, multimodal reasoning, and multi-agent orchestration. This is also our first API release! We’d love to hear your feedback. And we’re just getting started, super proud of the team behind it, and the larger models are training right now. 🚀 https://ai.meta.com/blog/introducing-muse-spark-meta-model-api/· Meta 推出 Muse Spark 1.1，最强模型 API 发布

<p>Excited to share what we’ve been building at Meta Superintelligence Labs!<br> <br> Toda…

#tw-jasonwei
📎 Jason Wei🕒 07-09 22:24🔗 rss.xcancel.com
行业资讯精选 85

Beautifully written piece by @FAbnousi about how AI for health might look like in the future The current data in health is limited because it only captures episodic clinical snapshots of what happens to our bodies The revelation is that there is so much latent knowledge in looking at regular changes in our body. Could be through lab tests or even proxy metrics like wearable data With AI democratizing the ability for people to understand their own health, we're moving towards a trend of individuals gathering health data around their bodies and leveraging AI to understand themselves Bryan Johnson is extreme but a good example of this trend Personally, I started getting Function Health blood tests every six weeks instead of the recommended six months to increase fidelity on how changes in lifestyle affect my body Of course i use AI to analyze the results and adapt, and it's been great It would be cool to something in this direction happen in a big way across the world And welcome to twitter @FAbnousi!· 未来健康领域的AI展望

<p>Beautifully written piece by <a href="https://rss.xcancel.com/FAbnousi" title="Freddy A…

#tw-jasonwei
📎 Jason Wei🕒 04-17 01:37🔗 rss.xcancel.com
行业资讯精选 82

Fun nine months! My first week i remember we had a long dinner in the cafeteria daydreaming about the cool research directions to pursue, then going to back to our desks to write a basic script to inference llama. Now we have a pretty complete stack and our first model is out 🥑· 从梦想初启到模型发布

<p>Fun nine months! My first week i remember we had a long dinner in the cafeteria daydrea…

#tw-jasonwei
📎 Jason Wei🕒 04-09 01:25🔗 rss.xcancel.com
行业资讯精选 82

Pinned: Becoming an RL diehard in the past year and thinking about RL for most of my waking hours inadvertently taught me an important lesson about how to live my own life. One of the big concepts in RL is that you always want to be “on-policy”: instead of mimicking other people’s successful trajectories, you should take your own actions and learn from the reward given by the environment. Obviously imitation learning is useful to bootstrap to nonzero pass rate initially, but once you can take reasonable trajectories, we generally avoid imitation learning because the best way to leverage the model’s own strengths (which are different from humans) is to only learn from its own trajectories. A well-accepted instantiation of this is that RL is a better way to train language models to solve math word problems compared to simple supervised finetuning on human-written chains of thought. Similarly in life, we first bootstrap ourselves via imitation learning (school), which is very reasonable. But even after I graduated school, I had a habit of studying how other people found success and trying to imitate them. Sometimes it worked, but eventually I realized that I would never surpass the full ability of someone else because they were playing to their strengths which I didn’t have. It could be anything from a researcher doing yolo runs more successfully than me because they built the codebase themselves and I didn’t, or a non-AI example would be a soccer player keeping ball possession by leveraging strength that I didn’t have. The lesson of doing RL on policy is that beating the teacher requires walking your own path and taking risks and rewards from the environment. For example, two things I enjoy more than the average researcher are (1) reading a lot of data, and (2) doing ablations to understand the effect of individual components in a system. Once when collecting a dataset, I spent a few days reading data and giving each human annotator personalized feedback, and after that the data turned out great and I gained valuable insight into the task I was trying to solve. Earlier this year I spent a month going back and ablating each of the decisions that I previously yolo’ed while working on deep research. It was a sizable amount of time spent, but through those experiments I learned unique lessons about what type of RL works well. Not only was leaning into my own passions more fulfilling, but I now feel like I’m on a path to carving a stronger niche for myself and my research. In short, imitation is good and you have to do it initially. But once you’re bootstrapped enough, if you want to beat the teacher you must do on-policy RL and play to your own strengths and weaknesses :)· 过去一年成为强化学习狂热者最大的感悟

<p>Becoming an RL diehard in the past year and thinking about RL for most of my waking hou…

#tw-jasonwei
📎 Jason Wei🕒 07-16 09:26🔗 rss.xcancel.com
行业资讯精选 82

New blog post about asymmetry of verification and "verifier's law": https://www.jasonwei.net/blog/asymmetry-of-verification-and-verifiers-law Asymmetry of verification–the idea that some tasks are much easier to verify than to solve–is becoming an important idea as we have RL that finally works generally. Great examples of asymmetry of verification are things like sudoku puzzles, writing the code for a website like instagram, and BrowseComp problems (takes ~100 websites to find the answer, but easy to verify once you have the answer). Other tasks have near-symmetry of verification, like summing two 900-digit numbers or some data processing scripts. Yet other tasks are much easier to propose feasible solutions for than to verify them (e.g., fact-checking a long essay or stating a new diet like "only eat bison"). An important thing to understand about asymmetry of verification is that you can improve the asymmetry by doing some work beforehand. For example, if you have the answer key to a math problem or if you have test cases for a Leetcode problem. This greatly increases the set of problems with desirable verification asymmetry. "Verifier's law" states that the ease of training AI to solve a task is proportional to how verifiable the task is. All tasks that are possible to solve and easy to verify will be solved by AI. The ability to train AI to solve a task is proportional to whether the task has the following properties: 1. Objective truth: everyone agrees what good solutions are 2. Fast to verify: any given solution can be verified in a few seconds 3. Scalable to verify: many solutions can be verified simultaneously 4. Low noise: verification is as tightly correlated to the solution quality as possible 5. Continuous reward: it’s easy to rank the goodness of many solutions for a single problem One obvious instantiation of verifier's law is the fact that most benchmarks proposed in AI are easy to verify and so far have been solved. Notice that virtually all popular benchmarks in the past ten years fit criteria #1-4; benchmarks that don’t meet criteria #1-4 would struggle to become popular. Why is verifiability so important? The amount of learning in AI that occurs is maximized when the above criteria are satisfied; you can take a lot of gradient steps where each step has a lot of signal. Speed of iteration is critical—it’s the reason that progress in the digital world has been so much faster than progress in the physical world. AlphaEvolve from Google is one of the greatest examples of leveraging asymmetry of verification. It focuses on setups that fit all the above criteria, and has led to a number of advancements in mathematics and other fields. Different from what we've been doing in AI for the last two decades, it's a new paradigm in that all problems are optimized in a setting where the train set is equivalent to the test set. Asymmetry of verification is everywhere and it's exciting to consider a world of jagged intelligence where anything we can measure will be solved.· 验证的非对称性和验证者法则

<p>New blog post about asymmetry of verification and "verifier's law": <a href="https://ww…

#tw-jasonwei
📎 Jason Wei🕒 07-16 08:59🔗 rss.xcancel.com
行业资讯精选 81

We don’t have AI self-improves yet, and when we do it will be a game-changer. With more wisdom now compared to the GPT-4 days, it's obvious that it will not be a “fast takeoff”, but rather extremely gradual across many years, probably a decade. The first thing to know is that self-improvement, i.e., models training themselves, is not binary. Consider the scenario of GPT-5 training GPT-6, which would be incredible. Would GPT-5 suddenly go from not being able to train GPT-6 at all to training it extremely proficiently? Definitely not. The first GPT-6 training runs would probably be extremely inefficient in time and compute compared to human researchers. And only after many trials, would GPT-5 actually be able to train GPT-6 better than humans. Second, even if a model could train itself, it would not suddenly get better at all domains. There is a gradient of difficulty in how hard it is to improve oneself in various domains. For example, maybe self-improvement only works at first on domains that we already know how to easily fix in post-training, like basic hallucinations or style. Next would be math and coding, which takes more work but has established methods for improving models. And then at the extreme, you can imagine that there are some tasks that are very hard for self-improvement. For example, the ability to speak Tlingit, a native american language spoken by ~500 people. It will be very hard for the model to self-improve on speaking Tlingit as we don’t have ways of solving low resource languages like this yet except collecting more data which would take time. So because of the gradient of difficulty-of-self-improvement, it will not all happen at once. Finally, maybe this is controversial but ultimately progress in science is bottlenecked by real-world experiments. Some may believe that reading all biology papers would tell us the cure for cancer, or that reading all ML papers and mastering all of math would allow you to train GPT-10 perfectly. If this were the case, then the people who read the most papers and studied the most theory would be the best AI researchers. But what really happened is that AI (and many other fields) became dominated by ruthlessly empirical researchers, which reflects how much progress is based on real-world experiments rather than raw intelligence. So my point is, although a super smart agent might design 2x or even 5x better experiments than our best human researchers, at the end of the day they still have to wait for experiments to run, which would be an acceleration but not a fast takeoff. In summary there are many bottlenecks for progress, not just raw intelligence or a self-improvement system. AI will solve many domains but each domain has its own rate of progress. And even the highest intelligence will still require experiments in the real world. So it will be an acceleration and not a fast takeoff, thank you for reading my rant· AI自改进将是渐进的

<p>We don’t have AI self-improves yet, and when we do it will be a game-changer. With more…

#tw-jasonwei
📎 Jason Wei🕒 07-01 03:06🔗 rss.xcancel.com
行业资讯精选 70

The most rewarding thing about working in the office on nights and weekends is not the actual work you get done, but the spontaneous conversations with other people who are always working. They’re the people who tend to do big things and will become your most successful friends· 夜班与周末办公的真正收获

<p>The most rewarding thing about working in the office on nights and weekends is not the …

#tw-jasonwei
📎 Jason Wei🕒 06-30 05:15🔗 rss.xcancel.com
行业资讯精选 62

I would say that we are undoubtedly at AGI when AI can create a real, living unicorn. And no I don’t mean a $1B company you nerds, I mean a literal pink horse with a spiral horn. A paragon of scientific advancement in genetic engineering and cell programming. The stuff of childhood dreams. Dare I say it will happen in our lifetimes· 当 AI 能创造活独角兽，我们就能说实现了 AGI

<p>I would say that we are undoubtedly at AGI when AI can create a real, living unicorn. A…

#tw-jasonwei
📎 Jason Wei🕒 06-29 05:01🔗 rss.xcancel.com
行业资讯精选 85

AI research is strange in that you spend a massive amount of compute on experiments to learn simple ideas that can be expressed in just a few sentences. Literally things like “training on A generalizes if you add B”, “X is a good way to design rewards”, or “the fact that method M is sample efficient means that we should create environments with this specific property”. But somehow if you find the correct five ideas and you really understand them deeply, suddenly you’re miles ahead of the rest of the field· AI 研究的秘密：关键想法的重要性

<p>AI research is strange in that you spend a massive amount of compute on experiments to …

#tw-jasonwei
📎 Jason Wei🕒 06-25 03:15🔗 rss.xcancel.com
行业资讯精选 85

My favorite thing an old OpenAI buddy of mine told me is, whenever he hears that someone is a “great AI researcher”, he just directly spends 5 minutes looking at that person‘s PRs and wandb runs. People can do all kinds of politics and optical shenanigans, but at the end of the day code and experiments don't lie. I checked out some of the diehard AI researchers and there are very few days where they haven’t launched an experiment· 如何辨别真AI牛人

<p>My favorite thing an old OpenAI buddy of mine told me is, whenever he hears that someon…

#tw-jasonwei
📎 Jason Wei🕒 06-22 04:37🔗 rss.xcancel.com
行业资讯精选 82

One way of thinking about what AI will automate first is via the “description-execution gap”: how much harder is it to describe the task than to actually do it? Tasks with large description-execution gaps will be ripe for automation because it’s easy to create training data and the value of automating them is huge, even if execution is non-trivial: - Fixing grammar mistakes in a long piece of writing - Submitting receipts for reimbursement - Training a model that achieves performance of X on a standard evaluation benchmark - Building an app where the UI is easy to check but requires a lot of moving parts in the backend Description-execution gaps tend to be small when the task is high-context and not technically challenging. The value of automating these is by definition smaller, and it’s harder to create data for them. For example: - Data processing scripts where the code to process the data is shorter and more precise than a natural language description - Running an ablation study in a high-context codebase that trains specialized models - Editing a video in a specific style (often easier to edit the video yourself than to describe how each little edit should be done) - Buying chinese groceries for my mom (she has very specific items and amounts, it's easier for her to go herself than to describe to me exactly the item, how to select the best fruit, etc) A bit similar to the discriminator-generator gap, but not exactly the same. Some things, like editing a video in a specific style, can have a large discriminator-generator gap but small description-execution gap

<p>One way of thinking about what AI will automate first is via the “description-execution…

#tw-jasonwei
📎 Jason Wei🕒 06-19 03:24🔗 rss.xcancel.com