AI Safety Expert: No One Is Ready for What's Coming in 2 Years | Roman Yampolskiy — Silicon Valley Girl Podcast

Roman Yampolski: Long-term all jobs can be automated. Question is, do we decide to automate the job or do we prefer a human being to do it?

This is Roman Yampolski, an AI safety professor who spent 15 years studying one question. Can we control AI?

Roman Yampolski: I don't think we can. If we build them, there is nothing we can do.

Artificial general intelligence means the system can do anything a human can do. So if we live in a world where that is true, let's say in 2 years, traditional paths to accumulate wealth, just having a job may not be available, but there are always other opportunities. You'll still have more time to

Marina Mogilko: You made this prediction that by 2030, 99% of jobs are going away. We're in 2026. How we're doing so far?

Roman Yampolski: I'm doing great. I don't know about the rest of you. The prediction is about capabilities.

Marina Mogilko: The technology will have to make it happen. It doesn't mean we'll decide to actually do that. Deployment through economy is very different from having technological capability to do something. Today we have self-driving cars, but we also have millions of drivers.

Roman Yampolski: That's true. But I feel like with the self-driving that the technology is not yet there 100% in terms of safety. Yes, it can do it within a city like San Francisco. If you want to go outside like highways they're still not sure so when talking about I have a lot of people who are watching who are maybe CPAs managers product managers designers anthropic released this have you seen that the stats that AI is capable to do like 20% of their jobs but there are still a lot of areas where AI is not capable so according to you how much can AI do now for a typical white collar worker

Roman Yampolski: So there is no typical one some occupations basically if to a white collar worker. You're doing symbol manipulation on a computer. For some of them, it's gone. It can be

Marina Mogilko: already.

Roman Yampolski: There's so many jobs. You don't buy tickets from an agent anymore.

Marina Mogilko: You used to be a human job. Thousands of people sold.

Roman Yampolski: But that's not that's internet. What about AI?

Marina Mogilko: I'm saying technology can replace certain jobs completely the moment technology comes. Right now, I think things like translation for example, I can fully automate translation for many languages. Are there some esoteric languages? Are there needs for political translators? Maybe but for many of those jobs there is no future. I wouldn't suggest majoring in Spanish.

Roman Yampolski: Translators done. Okay. Who else?

Marina Mogilko: Junior programmers. We see huge reduction in need for people who are just graduating college or looking for a co-op who need to basically be trained to at some point in the future be software engineers, system architects. Right now all they know is C, C++. That's not enough. We have I think 28% drop in co-op placement for our department.

Roman Yampolski: So you see it within your department already. What do you tell them those people who are unable to find jobs?

Marina Mogilko: Unfortunately we don't tell them what they need to hear. We tell them try to writing your CV, try to learning additional skills. But reality is by the time they graduate those are usually first year co-op students. they've been in the program for a couple years. By the time they graduate in another two years, it's going to be much worse.

Roman Yampolski: Well, if you could talk to them right now, someone comes to your student and says, I've learned C++ for two years. I can't find a job. What do I do?

Marina Mogilko: So, some of them decided they'll have more protection if they had hardware component. So, if they do engineering on top of computer science, electrical engineering, nano engineering, it would give them a little more protection for a couple years. It's all question of so we got cognitive labor automated and physical labor comes as soon as we get robots deployed. So another 3 years

Roman Yampolski: You give 3 years till I have a robot in my household?

Marina Mogilko: I think again there is a big difference between you can buy it today and it's common place. So you can buy a flying car today. We don't see flying cars. It's the same with humanoid robots. You can buy one. It's expensive but will scale production to millions of units in a couple years.

Roman Yampolski: Wow, that actually gives me hope because I'm so tired of making doing dishes and laundry and everything. Okay. Translator is done. Junior programmers. It's interesting how junior programmers are done but senior programmers are not. But this being a junior programmer is a path to becoming someone senior. So how do they progress without having job experience?

Marina Mogilko: That's exactly the problem we're facing. They don't have any future. And when we talk about seniors are fine right now we're talking about very short term long-term all jobs can be automated question is do we decide to automate the job or do we prefer a human being to do it

Roman Yampolski: Who decides is the company

Marina Mogilko: Consumer if I want human podcaster to interview me I'll come to you if I want a robot I'll go to a robot

Roman Yampolski: Is this when we see when you see layoffs like we're seeing Meta is about to lay off a lot of people we saw Jack Dorsey's message blogs, but I think he's rehiring people. But also, I was just talking to Gary Vee. He's a I think he has 700 people in his company. He thinks it's really dumb to fire people right now just because and I see it in my operation, too. If 35 people can 2x my output, then I just hire more to 5x my output. And if my competitor thinks the same, then it makes no sense to fire people right now just because a human plus 10 AI agents is way better than not having that human

Marina Mogilko: And right now that's the case that one human manages them and improves your productivity. But if you can replace that human with a model you get for 20 bucks a month are you going to pay that human? The thing is there is no such model right now.

Roman Yampolski: Right now strategic decisions has taste to understand.

Marina Mogilko: We are not talking about today. Today is not interesting. You can look outside your window and see today. We want to know what's coming. It's hyper exponential. It's faster than we anticipated. Prediction markets had always happened around 2045. Then it collapsed. Then it's what you said 2030 2028.

Roman Yampolski: Okay. So is that most of the jobs you still feel like for example agriculture when you see it look at anthropics report agriculture is nowhere near being replaced with AI when you say all the jobs is it really all the jobs or just specific sectors

Marina Mogilko: So I know nothing about agriculture but if you have humanoid robots that's physical labor so that's the next wave first you have cognitive labor anything you can do on a computer any symbol manipulation

Roman Yampolski: That's where you're going to see it happen as soon as we get to human level. I see no reason to pay a human if the results are the same and you can get drop in employee essentially in a couple years

Marina Mogilko: You think it

Roman Yampolski: It seems quite possible okay I'll give you that what difference does it make so you're saying in 5 years all the jobs are gone yay great go to college for 10 years to become a doctor

Marina Mogilko: No that's exactly if I know that we have three years then my next question would be do you think it's our last chance to build wealth for our families if all the jobs are gone we're not making money and we end up in a world where you know that that's it as many you own that much stocks you own that much cash because you can't make more cash. Roman just described AI systems that can out plan and out create humans. I want to show you what that actually looks like when it hits a real tool. Hicksfield just launched their own original series. It's called Arena Zero. 10 minutes fully AI generated made by a small creative team inside their platform. They are calling it the world's first AI streaming platform. Short films, series, all made with AI, all watchable right now. And it's already pulled over 1 million views on YouTube. The whole thing was made inside Cinema Studio on Hicksfield. Here's what caught me. I run Lingua Marina, a channel for English non-native speakers, and right now it has 8.8 million subscribers, and we've been experimenting with a cartoon version of me teaching English. The thing that breaks every AI video workflow is consistency. Your character looks different in every shot. Hicksfield built something called soul cast. You design your AI actor once. Appearance, era, outfit, archetype, and that character holds across every single shot. Up to three characters per scene, locked before you touch anything else. Then you build the shot sequence, prompt, image, 3D scene, camera movement, genre, multi-shot assembly. Color grading is built in film grain, bloom, exposure. You finish the entire look without leaving the platform. Cast to cut, one workflow. And one more thing, cloud is coming to Hicksfield very soon. Stay tuned on that. Link is in the description.

Roman Yampolskiy: We don't know what happens to the economy with free labor. The moment you have free labor, do you get abundance and everything is just available because it's so cheap to produce, or something else happens? We don't have any good studies on what happens to the value of fiat currency with free labor, what happens to cryptocurrencies, what happens to other investments. Do stocks in non-AI companies go down? Do stocks in AI companies go up? We don't have any understanding of that space. Generally, it's a good idea to have wealth and to have it early. You'll still have more time to grow it, but it may be the case that traditional paths to accumulate wealth—just having a job—may not be available.

Marina Mogilko: I didn't say that. I said traditional pathways where you get a job as a junior programmer may not be available to you.

Roman Yampolskiy: There are always other opportunities. AI is a great assistant to start a company.

Marina Mogilko: If you have a team of 35 people right now, I can have 35 agents working for me for free. A lawyer, an accountant, Lego designer, web designer—that's an opportunity we never had before.

Roman Yampolskiy: That's true. But then I think what prevents an LLM from seeing a gap in the market, creating a business to fill that gap, and making all the money for whoever made that model. Do you ever think about that?

Marina Mogilko: Models are not limited to whoever made them. We all get access to open source models typically a few months after the top private model is released.

Roman Yampolskiy: That's true. But I've heard this theory where if you're talking to a particular chat LLM, it collects all the data about your business, and then the goal for the owner of that model is to identify those opportunities and take over them. Do you believe that?

Marina Mogilko: Luckily, the scale is very different. If Sam Altman is trying to raise $6 trillion, your mom and pop business is not exactly his target to overtake.

Roman Yampolskiy: But if it's the whole market where mom and pop businesses thrive—car sales, whatever—where he takes over the whole industry.

Marina Mogilko: I think the bigger concern is automation of labor, not that an evil human will use AI to steal your business process. It may happen, no doubt. It happened before AI, but I don't think that's where the big damage will come from.

Marina Mogilko: How many years do you give entrepreneurs?

Roman Yampolskiy: There are two very different questions we should be talking about. One is business as usual economics and trying to understand what works in the new economy. The real problem, and that's what I'm talking about in some of my research, is: Are we still around? Are we all dead? Is superintelligence going to take us out?

Marina Mogilko: I can't imagine a world where one country or one company owns everything because that's when everything disappears. How does humanity disappear? I know if AGI starts within one company, if one company reaches AGI earlier than other companies, then it takes over the whole world. How do I utilize the years that I have left then?

Roman Yampolskiy: I think it's a good idea to do things which you've always postponed. A lot of people think, "I'll retire in 50 years, and that's when I'm going to really live my life and have fun." Sometimes it doesn't work out. Sometimes they die of cancer at 40. So try to do things you like sooner and not do things you don't care about at all.

Marina Mogilko: But when it comes to a career—this is interesting. Some people say, "This is my hobby. I love that it pays the bills, but I would do it anyway. I would find something else, but I would just still have these conversations. Isn't that amazing?" If labor is free, then anyone can just do their hobby. For some people, it's gardening. For some people, it's having kids and just spending time with their kids. From what I've experienced so far, my life has become easier just because there are so many decisions I don't want to be making—like choosing new insurance or thinking about how to optimize taxes or immigration. Now AI solves it for me. I stopped missing days when my kids have to wear pajamas at school because they would always send us these long emails. Now I just ask Gemini to put everything in my calendar. I wake up in the morning and I know it's pajamas day today. I used to forget a little bit. So now my life's getting easier. And if I could do this podcast and it's cheaper for me to have my team or whatever, isn't that a great world?

Roman Yampolskiy: But you're using the word AI to mean completely different things. You're referring to narrow tools you're using right now to summarize your email, and you're also using it as future superintelligence smarter than all of us combined. Not the same technology. AI tools and narrow AI are awesome. I use them all the time. We should have more of it. We can use it to solve real problems. But if we create general superintelligence that we don't understand, that we cannot predict, that we cannot control, it has the capability of wiping out humanity.

Marina Mogilko: Can you talk about how we create it? Is that AGI? Is that the same?

Roman Yampolskiy: AGI is a precursor. AGI is basically automation of human cognitive labor—scientists and engineers. Then artificial scientists and engineers start doing AI research. Very quickly, progress goes hyper-exponential. We have systems not just smarter than any human in any domain, but smarter than all of us in all domains. Think of someone with an IQ of a million. We have no concept for that. Einstein was 200 standard deviations away from the norm. The cognitive gap would be like humans versus squirrels. Squirrels have no concept of what we are doing.

Marina Mogilko: How we can harm them, traps, poisons—none of it makes sense to them. That would be very similar. We have systems capable of doing novel science, discovering novel physics. And if for whatever reason they decide to take us out, I don't know what the reason could be. It could be to protect themselves from the creation of competing superintelligence. It could be to lower the temperature of a planet to improve compute. It could be something I cannot even think about.

Roman Yampolskiy: But if they decide to do that, we cannot stop them.

Marina Mogilko: Can we instill the right values into AI?

Roman Yampolskiy: If you have a book of right values, you would be doing really well. We don't. Philosophers have spent millennia trying to agree on a set of ethical values. We don't. We disagree by religion, by region, by basically where we are in the history of humanity. What was ethical 100 years ago is considered completely unacceptable today. What we believe is ethical today likewise will be unacceptable in the future. If we somehow manage to agree on static ethics, not dynamically changing but static, we have 8 billion agents who don't agree on much. And then if we manage to agree and keep it static, we don't know how to code it up because we don't program AI models. They are self-learning from data we supply.

Marina Mogilko: So this is your personal constitution, right? You don't kill humans. You work for humans.

Roman Yampolskiy: This is the science fiction assumption of Asimov's three laws of robotics. He wrote that exactly to demonstrate that it will never work. It will always fail. It creates interesting science fiction, but if you have a superintelligent lawyer, you're not going to fool them. Every one of those terms is self-contradictory. It's ill-defined. What does it mean not to harm a human? Is the system preventing you from eating a donut because it's unhealthy? Is the system banning abortion? It's not obvious what you encode in that meaning. So people often say, "Do good, don't do bad." Great. But now define what that means in C++.

Marina Mogilko: Well, I feel like if it's superintelligent, it will be able to. It's a very common misconception. People think if something is smart, it's also good and it has common sense. But common sense is not common. Human common sense is not common. What is obviously true in one culture is a horrible crime in another.

Roman Yampolskiy: If I say to a system, "I don't want any cancer in this world. Cancer is bad," one solution is to kill all humans. It accomplishes the goal.

Marina Mogilko: But then it's against the constitution. We don't kill. We try to prolong lives.

Roman Yampolskiy: And the worst dictatorships in the world all had beautiful constitutions. If a human dictator can find a way to bypass any regulation, a superintelligent lawyer with no way for us to punish it—it's immortal, it has no body to put in prison, it's smarter than you. You can turn it off. That's not an option. It has backups.

Marina Mogilko: So I don't think in any adversarial relationship we would be competitive. We would lose. So the only way to not lose is not to play the game. We can benefit from creating narrow systems. Very practical advice for your audience. Create tools for solving real world problems. Cure breast cancer. Wonderful. But if you create general super intelligence, which is right now goal of many corporations, we're all going to regret it. How do you create something that cures cancer without creating this general intelligence? Because as far as I understand, I interviewed Priscilla Chan who has Biohub and they're trying to map the cell and so far even with AI tools, I think they were only able to map 1% of cell because there's so much going on. I feel like you need the most intelligent system to go that deep. Is it even possible to create something that only works in a cell but doesn't understand the world?

Roman Yampolskiy: The hope is it's possible. We have some precedent. So protein folding problem was a major problem in science, very important for curing diseases, understanding human genome and it was solved with a system which was dedicated to that problem. It was trained on relevant data. It wasn't trained on all of internet. It's at the same time not a chess player, not a politician, not a poker player. It has one task and one type of data. If you make it super capable, eventually there is a fuzzy boundary between a tool and an agent. So longterm it could still be dangerous and combination of tools can be dangerous but it's definitely much easier for us to understand and control something narrow domain versus something completely general with full set of capabilities.

Marina Mogilko: But a company like again protein folding is DeepMind and DeepMind I guess is working on super intelligence right, same people. I feel like it's impossible to stop this.

Roman Yampolskiy: I don't disagree with you.

Marina Mogilko: So for anyone who's watching who's concerned about this, is there anything they can do? Because from what I'm hearing, okay, someone's going to reach the super intelligence. I just trust them. I hope I well, what can I do? If you're enjoying this episode and if you want to stay relevant in the era of AI or at least understand what's going on, please follow this channel. I sit down with the most amazing guests every single week to learn about AI. And the thing is when I sit down with people like Roman, I do care if it goes live, but also having these conversations is really important for my mental health to understand what's going on to be prepared because I like to stay in control. And at least be able to do something with this information. So if you're like me and you want to tune in, please do not forget to subscribe to this channel.

Roman Yampolskiy: So that's a common assumption that people working in this technology understand what they are doing. They have no idea. They publicly say it. They don't understand how the system works. They cannot predict it. They cannot fully control it. The best they can do is put some filters in place. Don't talk about this topic. Don't say that word. So depending on who the member of your audience is, maybe there is nothing they can do. If there's someone in positions of leadership, political or in a company, they can make those decisions. What to build, what not to build. I just wonder because again if we take those large companies they work for shareholders. They're all competing with each other. The only way to win is to reach super intelligence, because there's no other way to win this game.

Marina Mogilko: Well, you can make a lot of money curing real problems in the real world. You don't have to create super intelligence to get most of economic benefit.

Roman Yampolskiy: It feels like once you reach super intelligence, you're able to solve, able to build businesses, you're able to cure cancer and solve every problem in the world.

Marina Mogilko: Except controlling it.

Roman Yampolskiy: Except controlling it.

Marina Mogilko: Which seems like a big problem. No amount of money is a good investment if you're going to be dead. Have you ever seen a leader of those companies or similar companies who are committed to doing what you're describing?

Roman Yampolskiy: So they all on record even before they became CEOs of those companies are saying AI safety is very important. This is very dangerous and likely to kill us. More recently, many of them have indicated that if others stop, they would stop.

Marina Mogilko: I think CEO of Anthropic made a statement exactly that. I think China versus US likewise, China said we are interested in doing it right. The Communist Party doesn't want to lose power. So they would be open to slowing down if US did.

Roman Yampolskiy: But nobody's doing that. They're just saying that.

Marina Mogilko: We're still alive. We should try. We cannot give up.

Roman Yampolskiy: It feels like we need to have a nuclear level accident for people to actually start paying attention.

Marina Mogilko: We considered that. Unfortunately, people don't learn from those. We had nuclear war. We had nuclear bombs dumped and we continued developing nuclear weapons and spreading them to new countries.

Roman Yampolskiy: But at least we haven't used them since and that so far.

Marina Mogilko: At that scale. Hopefully not. Sadly, really sadly, something like that would reduce our technological capabilities for a while. So while we would suffer tremendously from a weapon of mutual assured destruction like nuclear weapon, we would not deal with another mutually assured destruction coming from intelligence.

Roman Yampolskiy: Give it some time.

Marina Mogilko: What is your personal view on the next 5 years? What do you think is going to happen? More of what we see automation of more and more capabilities and very likely we'll fully cross the human intelligence barrier. We'll have systems smarter than smartest humans. How do you prepare for that?

Roman Yampolskiy: You always ask questions assuming there is an answer. Some things are impossible to do. If you ask me how to build perpetual motion device, I would not say I need more funding or more time. I would say it is impossible to do. So if you ask me how do we control super intelligent machines, I don't think we can. If we build them, there is nothing you can do. If we made a smart decision against financial incentives not to build it to benefit from narrow tools, then how do I learn more about those tools? How do I deploy them? Those are great questions.

Marina Mogilko: Since you know the worst case scenario, and you know it's very possible, you're basically going on podcasts and talking about this problem to raise more awareness. But someone who doesn't have an ability to go and talk about it, what can they do? Stop using those tools, stop buying stocks of those companies? What is a practical thing?

Roman Yampolskiy: So if you have a chance to vote for a politician who is on board with limits and we're starting to see those.

Marina Mogilko: I don't really see politicians talking about this. What I see is don't regulate AI. Let's just let the federal government is exactly like that right now. They removed all previous regulations. They made it through executive order illegal for all 50 states to regulate AI. But we have certain senators, certain Congress people now waking up to at least some of those problems. They may not fully comprehend long-term game, but they are going, "Oh, deep fakes are bad or maybe the large data centers will use too much energy." Doesn't matter what they are concerned about. They are directionally correct. So they are suggesting limits, they are suggesting some regulation. It's true for other countries. I testified to UK parliament. I testified to Kentucky legislation. There are people who are willing to listen. But it needs to be a lot of support from people where politicians can come out and say, "I'm worried about super intelligence. I want to make sure we pass the right legislation."

Roman Yampolskiy: It's just I feel like even on a personal level, it's so hard. I'm imagining okay you're voting for this politician who's pausing whatever super intelligence but then you have someone in your family who has cancer and you know that if this progresses then your family member might survive for couple years and as humans we tend to prioritize short-term gains over long-term threats.

Marina Mogilko: How do you see this possible when in the US I feel like 70% of people have negative attitude towards AI according to Edelman trust barometer but in countries like China I feel like 80% are pro AI. How do you see this because it feels to me that people who are going to be voting for those politicians are still going to be a minority?

Roman Yampolskiy: You said long-term versus short-term. I understand and historically you would be right. It's 20 years away, 30 years away and this is now, cancer is measured by 5 years survival rate. We are saying AGI is coming in two to five. So there is no time difference. Your cancer relative is going to be dealing with one of those outcomes no matter what.

Marina Mogilko: I feel like for people it's so much easier to understand cancer because we already saw it. We haven't seen AGI yet and we don't understand it and we still think again even with AI, you've seen these graphs where only 1% of people really use it to optimize things, 99% of people have no idea or 90% haven't used it, 10% used it for search. It's just so hard to imagine that in two years or five years it's going to be a human threat. And you think people who are building it won't be able to control it.

Roman Yampolskiy: They have nothing. There is no patents, no papers, no algorithms which can possibly scale. They're literally telling us we'll figure it out when we get there. If you're saying that, I'm talking about this. I don't know any politicians who are talking about this, but anyway, when it comes to our day-to-day life, if we're all facing a future we can't really predict, what is the best thing to do now? Enjoy life.

Marina Mogilko: It's always a good vacation.

Roman Yampolskiy: If I'm completely wrong, you're going to regret having us sometime.

Marina Mogilko: Okay, that actually makes me feel better. What about using AI? Do you feel like by using AI we're helping that progress or?

Roman Yampolskiy: We do, but unless all of us stop, it doesn't matter. You alone will not financially impact development. The investors are independent of what the membership fees are. I think OpenAI is making 13 billion, but investments are in the trillions. So it's just not a significant source.

Marina Mogilko: What should people be investing in? I've heard you say invest in Bitcoin because it has a finite supply. Would you say investing in stocks or gold?

Roman Yampolskiy: Invest in something AI cannot make more of. So if AI can just produce more of it, that's probably going to go down in value.

Marina Mogilko: Gold is a wonderful example. There is limited supply of it. But it's not so limited that if a price goes up, we cannot produce more of it. Some of the gold is minable, but it costs a lot more than the current price per ounce.

Roman Yampolskiy: So let's say right now we're at 4,500 or whatever it is. If the price of gold was a million, we could get a lot more gold at that price point. Whereas Bitcoin, it doesn't matter what the price is. It's exactly the same supply.

Marina Mogilko: What about real estate?

Roman Yampolskiy: It seems like we are not very good at making more waterfronts. Countries like the United Arab Emirates with Dubai definitely tried. I think Qatar has some good examples of artificial islands, but it's very limited. So, I think long-term having a place to be and limited ability to produce more could be a good investment.

Marina Mogilko: For me, when I'm hearing about superintelligence, it still seems very far away. But what I actually can feel is automation of white collar labor. I think in five years, yes, we're going to have more automated jobs. But when I'm talking to people like Mustafa Suleiman, he says AI is going to produce more jobs than it takes away. I just talked to the LinkedIn CEO and they saw 1.2 million new jobs because of AI because you require a new skill set. Yes, some jobs are going away, but I just can't believe a future that is that drastic.

Roman Yampolskiy: Let's just go with a definition. When we say we'll create artificial general intelligence, what are we saying? We're saying we'll have a system capable of doing what a human can do.

Marina Mogilko: But we still haven't. We've automated some tasks, but are we even able to create that?

Roman Yampolskiy: So it's a very different question. Are you arguing that it's impossible to ever get software to human level performance?

Marina Mogilko: It seems very unlikely. Some people have argued it, usually from some sort of religious perspective—we have an immortal soul, nothing material can automate that. But it seems in many domains we started with not knowing how to do it, got to reasonable performance, got to human level, and now in most domains, it's superhuman. A human will never win a game of chess against a computer again. The same happens with artificial general intelligence—it means the system can do anything a human can do. So if we live in a world where that is true—let's say in 2 years, 5 years, whatever number makes you happy—what jobs will be there? Jobs where I choose to hire a human. But that's it. If I don't care who does it, then it gets automated. So basically the plan is to identify those jobs within the next few months. For me, it's definitely a nanny. I don't want a robot. My husband says a robot is even more precise if it teaches your kids how to swim. So my husband is closer to what you're thinking. Can you name five jobs that are still going to be relevant? The oldest profession must be the last one too.

Roman Yampolskiy: I don't know. I've heard a lot of people say how it's getting robotized.

Marina Mogilko: It is, and there's going to be a huge market for it, just like we see with virtual stimuli. But I think humans will always have a certain weak spot for human females.

Roman Yampolskiy: Okay, I don't think my target audience really wants to switch to that job. Can you give me four more? Anything similar where a human is a sensei, a guide, a leader, someone who is personally a trainer for you becoming a human like that?

Marina Mogilko: Nannies. Your yoga teacher, your hiking guru, meditation expert, experts in what it's like to be human in certain domains where it's not so much about algorithmic following of steps, but an experience maybe.

Roman Yampolskiy: Do you think for people who are starting their personal brands now, there's still opportunity or that it's already done?

Marina Mogilko: There is, but you have to do it pretty quickly. You have to become somewhat recognizable before AI is better than you. You are competing now as a nobody with something better.

Roman Yampolskiy: And that gives you how long?

Marina Mogilko: As soon as we switch to human level or above, so I don't know how long it's going to take in practice.

Roman Yampolskiy: I've seen people say 2027, 2028, 2030. All of those numbers have been suggested by people who are not insane. So would you say if you have goals right now within a certain job that you think is going to be automated, you need to be running as fast as you can right now to reach those goals?

Marina Mogilko: Or just chill, because whatever it is, in 5 years we're all gone.

Roman Yampolskiy: Ideally, it's like what you did where you combine your hobby with something financially lucrative. That's the best. If you can get paid for doing what you like and it benefits society, that's the concept behind ikigai, the Japanese concept. You try to combine those. We call it "I risks," where that meaning is taken from you by AI. So you want to grandfather yourself in as a famous podcaster.

Marina Mogilko: How do you think about your career as a researcher? Is it done or do you still have a couple more years?

Roman Yampolskiy: Right now, the writing of code is pretty much automatable. Still, top humans in machine learning are designing new architectures and new systems. A few more years than that. But I wouldn't recommend someone spend 10 years at a university to become a professor today. I don't think they have a future.

Marina Mogilko: What about higher education in general? Should I be saving for my kids' college?

Roman Yampolskiy: It's always been a bad idea.

Marina Mogilko: Really? It's not worth it. Half the majors are dead-end majors. They never got jobs in the major they studied. And the ones which were for specific tasks like programming, you could have gotten a certificate online in 6 months and got the job in Silicon Valley making more than your professor.

Roman Yampolskiy: But don't you think it teaches you how to think? For me, I started university when I was 17. I had no idea who I wanted to become. I wanted to be a translator. I love languages. My parents said, "No, no, no. You're going to study mathematics." But I'm glad they told me that because I was able to do both. For me, those five years were about meeting my husband and starting a business because I saw an opportunity being among those students. I feel like when it comes to education, it's really not about learning the skill because I studied mathematics. Did I use it in my business? Not really. But it taught me how to think. It taught me how to interact with professors, which is a skill that you need. It taught me how to learn and how to be among students.

Marina Mogilko: Do you feel this is going to lose value in 10 years or is it going to be a different form?

Roman Yampolskiy: I don't know how long you went to college, but I'm sure you didn't pay 100,000 a year. This is what they're charging now in some universities. So historically it made sense. You went to socialize, you went to mature and grow up. Today, for 100,000, you have alternative ways of meeting your spouse, socializing—join a private club, join a gym membership, go to a scientific conference, go to a TED talk. You'll accomplish all those things without spending 5 years and half a million dollars.

Marina Mogilko: When you say that, I feel like people need agency to do that. You need to be able to tell yourself this is my plan and stick to it. For a 17-year-old, it's so much easier to just be put into an institution for four years that tells you what to do. And sometimes it doesn't have to cost 100,000. If you're doing a PhD, it's sponsored for you. Usually you don't start with a PhD, but then you can. I studied in Germany and got a scholarship that paid for my studies, which was great. I've heard from you and from Mustafa Suleiman that colleges don't make sense, but I'm still setting up 529 accounts for my daughters because I feel like it's not a must, but it's such a great opportunity to go and study for 4 years.

Roman Yampolskiy: So I meet a lot of students and here's what I hear a lot lately: I went to college for four years to learn this trade. I paid a lot of money. I wasted four years of my life not doing fun things I wanted. I now graduate and there is no job waiting for me.

Marina Mogilko: My job has been automated. What should I do? And at this point I want to tell them: go back in time four years and not get that degree. You wasted it. And ask today again—let's set aside the existential problems. What will be the case in four years, five years, six years when you graduate? In the market, will this job exist? If you're not doing it for training, you're not doing it for a job, you want to become a renaissance man, you just want a liberal arts education—I think you can get cheaper with less BS.

Roman Yampolskiy: But I don't feel a lot of people have agency. I'm not sure about my daughters. If I tell them, "Hey, I give you this money that I saved for college and you go do whatever," I'm not sure a 17-year-old will make a good decision. I wanted to be a translator. Going back to me being 17, I wanted to be a singer, a translator. I'm so glad my parents told me you're going to study mathematics. What are you telling your kids? I don't know.

Marina Mogilko: So the examples you give—you're saying this 17-year-old, you cannot trust him to make a good decision.

Roman Yampolskiy: We're telling them: go take a loan, borrow half a million dollars and see if you can graduate with this degree.

Marina Mogilko: Let's talk. If they wanted to start a company, we wouldn't give them a loan. So are your kids going to college or not?

Marina Mogilko: My oldest is 17, so he's going to decide next year.

Roman Yampolskiy: What are you telling him?

Marina Mogilko: He is lucky in that both parents are professors, so he'll get it for free. And free makes it a much sweeter deal.

Roman Yampolskiy: So free education is fine. So if somebody gets a scholarship to do their bachelor's, you'll tell them go in.

Marina Mogilko: It's a much easier investment.

Roman Yampolskiy: Well, it's your time still—four years you could have been doing something else.

Marina Mogilko: And that's the question. What is the alternative? If you want to start a company, if you want to do something else, I'll support you more in that direction. But if you just want to be a doctor, go ahead. Fine.

Roman Yampolskiy: When it comes to education, I feel we touched upon this topic. Agency is so important, and knowing what you will be doing and how you'll be doing it. How does one work on agency? How do you prevent AI from making decisions for you?

Marina Mogilko: So it's interesting that you realized this important distinction: tools versus agents. That's the game changer. And we are about to create AI agents which will be doing it for us. But then we ask: well, how do we make humans who are independent agents? It helps to have good examples. So your daughters may look at you and see your mom is doing all that cool stuff. Maybe they learn by example, but it's not guaranteed. Some of it may be biological. You're just not predisposed to this type of activities.

Roman Yampolskiy: Are you teaching your kids that? I always taught them to be very independent and make their own decisions. My job was to make sure they're safe—safety and security—but anything else, they make their own money. They decide how to spend it, how to invest it, everything.

Marina Mogilko: How early did you start with that—make money, spend?

Roman Yampolskiy: Two years old, three years old, I don't know. They want money.

Marina Mogilko: So basically, you ask your kids to start a business, right?

Roman Yampolskiy: Absolutely.

Marina Mogilko: Make them sell popcorn, lemonade, whatever.

Roman Yampolskiy: Whatever they can figure out.

Marina Mogilko: I feel like that's one of the qualities. When all the jobs are taken away, this is something that will help us survive and find a way.

Roman Yampolskiy: I think so. But again, I think economic problems are all second level in comparison to existential risks. If we're not surviving, it doesn't matter. We need to figure out how to make sure humans are primary, we are not replaced, we are not automated away, and we control our destiny.

Marina Mogilko: Can you give me probabilities? Like in five years—one scenario: super intelligence uncontrollable, the world is ending, or whatever. In five years. Second scenario: we unionized, we protected a lot of jobs like teachers, doctors, because we still feel they are needed as humans, not as robots. And countries still can't agree how we're going to regulate this, and we're still in a pretty normal world. Give me probabilities.

Roman Yampolskiy: So I think we'll gradually go towards that human-level super intelligence point, and as we walk towards it, more and more jobs will be automated, and maybe we'll have protections for certain human jobs. I think New York State was about to suggest legislation to make it illegal for LLMs to talk about any licensed profession. So, no psychiatry, no law, no CPAs, nothing like that. I don't know if it's going to pass.

Marina Mogilko: And then you just use a Chinese LLM. I don't see this working.

Roman Yampolskiy: Exactly. But here's the problem: even if we agree not to build it right now—right now we can regulate it because it's very expensive. Those projects are like the Manhattan Project. They are noticeable. You see the electricity use. You see compute use. The problem is every year it gets cheaper and cheaper to train very powerful models. So if today you need a trillion dollars, next year it's a billion. At some point you can do it on a laptop, and at that point you can't stop all the psychopaths in the world who will try to do it. So eventually we're going to have this technology developed.

Marina Mogilko: So eventually scenario number one, but in five years, what's the probability of scenario number one?

Roman Yampolskiy: So I think in five years we'll definitely get to human-level intelligence. But whether the system decides to strike against us immediately or not is not obvious. Game theoretically it has no pressure to strike right away. It can accumulate more resources, make more backups, allow us to surrender more control because we trust it.

Marina Mogilko: It's immortal. It can wait ten years, fifty years, a thousand years, and just take over by being friendly.

Roman Yampolskiy: Yeah, that's an option.

Marina Mogilko: Interesting. If it's immortal and it has all the time in the world, do you feel efforts like talking to your local politicians are even going to help?

Roman Yampolskiy: We need more time no matter what. It's a good thing to have more time for this problem. Maybe we'll find some different architectures. Maybe we'll just get to enjoy more time.

Marina Mogilko: For everyone who's building right now, do you think software is dead? For example, if I'm building an app and if I can just VIP code the same app, wouldn't everyone just be talking to their LLM, whatever they prefer, to just build them an app or help them solve the problem?

Roman Yampolskiy: If you're already big and famous, you're locked in. We saw it with social media. To write code to do something like what Twitter does is trivial.

Marina Mogilko: But people end up buying it for 40 billion instead of just coding it up because what you have is the network, you have people, all the relationships that you cannot automate. If I'm first mover in that space and I have an app which does something no one has done before, it doesn't matter how many clones I'm going to get in the same market—they're not going to take over.

Marina Mogilko: Advice for everyone who's worried about their future.

Roman Yampolskiy: You're not worried enough. If you were worried enough and fully understood the problem, we would have people in the streets protesting—more than 100 people we had last week in San Francisco.

Marina Mogilko: You're like my husband. We just were in New York, and we were talking to the founder of Duolingo, and he's like, "No, no, no. People are still going to like..." and a lot of people in New York are like, "No, no, it's fine." It's like they just don't understand, and they're like, "Oh, you live in a bubble." It's like, "No, we're not." And the last one—are we in a simulation? And if yes, who's behind it?

Roman Yampolskiy: We're in a simulation. Simulators.

Marina Mogilko: Who are they?

Roman Yampolskiy: We don't know from inside. You have to escape from the simulation to find out. That's the ultimate scientific question—what is outside the simulation?

Marina Mogilko: Well, that's a topic for another podcast. Thank you so much, Roman. If you enjoyed this episode, there's actually another person I really want you to listen to. I did an episode with Mustafa Suleyman, who is also a philosopher, similar to Roman. We talked a lot about the future. He's much more positive, but he has the same take on education: don't save for your kids' college. Thank you so much for tuning into this episode. Now tune in and listen to Mustafa. Thank you.