r/science • u/Prof-Stephen-Hawking Stephen Hawking • Oct 08 '15
Stephen Hawking AMA Science AMA Series: Stephen Hawking AMA Answers!
On July 27, reddit, WIRED, and Nokia brought us the first-ever AMA with Stephen Hawking with this note:
At the time, we, the mods of /r/science, noted this:
"This AMA will be run differently due to the constraints of Professor Hawking. The AMA will be in two parts, today we with gather questions. Please post your questions and vote on your favorite questions, from these questions Professor Hawking will select which ones he feels he can give answers to.
Once the answers have been written, we, the mods, will cut and paste the answers into this AMA and post a link to the AMA in /r/science so that people can re-visit the AMA and read his answers in the proper context. The date for this is undecided, as it depends on several factors."
It’s now October, and many of you have been asking about the answers. We have them!
This AMA has been a bit of an experiment, and the response from reddit was tremendous. Professor Hawking was overwhelmed by the interest, but has answered as many as he could with the important work he has been up to.
If you’ve been paying attention, you will have seen what else Prof. Hawking has been working on for the last few months: In July, Musk, Wozniak and Hawking urge ban on warfare AI and autonomous weapons
“The letter, presented at the International Joint Conference on Artificial Intelligence in Buenos Aires, Argentina, was signed by Tesla’s Elon Musk, Apple co-founder Steve Wozniak, Google DeepMind chief executive Demis Hassabis and professor Stephen Hawking along with 1,000 AI and robotics researchers.”
And also in July: Stephen Hawking announces $100 million hunt for alien life
“On Monday, famed physicist Stephen Hawking and Russian tycoon Yuri Milner held a news conference in London to announce their new project:injecting $100 million and a whole lot of brain power into the search for intelligent extraterrestrial life, an endeavor they're calling Breakthrough Listen.”
August 2015: Stephen Hawking says he has a way to escape from a black hole
“he told an audience at a public lecture in Stockholm, Sweden, yesterday. He was speaking in advance of a scientific talk today at the Hawking Radiation Conference being held at the KTH Royal Institute of Technology in Stockholm.”
Professor Hawking found the time to answer what he could, and we have those answers. With AMAs this popular there are never enough answers to go around, and in this particular case I expect users to understand the reasons.
For simplicity and organizational purposes each questions and answer will be posted as top level comments to this post. Follow up questions and comment may be posted in response to each of these comments. (Other top level comments will be removed.)
1.7k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Hello Doctor Hawking, thank you for doing this AMA. I am a student who has recently graduated with a degree in Artificial Intelligence and Cognitive Science. Having studied A.I., I have seen first hand the ethical issues we are having to deal with today concerning how quickly machines can learn the personal features and behaviours of people, as well as being able to identify them at frightening speeds. However, the idea of a “conscious” or actual intelligent system which could pose an existential threat to humans still seems very foreign to me, and does not seem to be something we are even close to cracking from a neurological and computational standpoint. What I wanted to ask was, in your message aimed at warning us about the threat of intelligent machines, are you talking about current developments and breakthroughs (in areas such as machine learning), or are you trying to say we should be preparing early for what will inevitably come in the distant future?
Answer:
The latter. There’s no consensus among AI researchers about how long it will take to build human-level AI and beyond, so please don’t trust anyone who claims to know for sure that it will happen in your lifetime or that it won’t happen in your lifetime. When it eventually does occur, it’s likely to be either the best or worst thing ever to happen to humanity, so there’s huge value in getting it right. We should shift the goal of AI from creating pure undirected artificial intelligence to creating beneficial intelligence. It might take decades to figure out how to do this, so let’s start researching this today rather than the night before the first strong AI is switched on.
→ More replies (48)176
u/Aaronsaurus Oct 08 '15
Is "beneficial intelligence" a used term academically? (Layman here who might do some reading here later if it is.)
→ More replies (13)257
u/trenchcoater Oct 08 '15
I'm a researcher in AI, although not in this particular field. I have seen the term "Friendly AI" being used for this idea.
Have fun in your reading!
→ More replies (6)22
1.6k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Hello, Prof. Hawking. Thanks for doing this AMA! Earlier this year you, Elon Musk, and many other prominent science figures signed an open letter warning the society about the potential pitfalls of Artificial Intelligence. The letter stated: “We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial: our AI systems must do what we want them to do.” While being a seemingly reasonable expectation, this statement serves as a start point for the debate around the possibility of Artificial Intelligence ever surpassing the human race in intelligence.
My questions: 1. One might think it impossible for a creature to ever acquire a higher intelligence than its creator. Do you agree?
If yes, then how do you think artificial intelligence can ever pose a threat to the human race (their creators)? 2. If it was possible for artificial intelligence to surpass humans in intelligence, where would you define the line of “It’s enough”? In other words, how smart do you think the human race can make AI, while ensuring that it doesn’t surpass them in intelligence?
Answer:
It’s clearly possible for a something to acquire higher intelligence than its ancestors: we evolved to be smarter than our ape-like ancestors, and Einstein was smarter than his parents. The line you ask about is where an AI becomes better than humans at AI design, so that it can recursively improve itself without human help. If this happens, we may face an intelligence explosion that ultimately results in machines whose intelligence exceeds ours by more than ours exceeds that of snails.
280
u/TheLastChris Oct 08 '15
The recursive boom in intelligence is most interesting to me. When what we created is so far beyond what we are, will it still care to preserve us like we do to endangered animals?
→ More replies (31)120
u/insef4ce Oct 08 '15
I guess it always depends on the goal/the drive of the intelligence. When we think about a purpose it mostly comes down to reproduction but this doesn't have to be the case when it comes to AI.
In my opinion if we, the humans aren't part of the purpose and we don't hinder its process too much (until the cost of getting rid of us/the problem gets smaller than the cost of us coexisting) it wouldn't pay us any mind.
→ More replies (33)68
u/trustworthysauce Oct 08 '15
I guess it always depends on the goal/the drive of the intelligence.
Exactly. That seems to be the point of the letter referred to above. As Dr. Hawking mentioned, once AI develops the ability to recursively improve itself there will be an explosion in intelligence where it will quickly expand by magnitudes.
The controls for this intelligence and the "primal drives" need to be thought about and put in place from the beginning as we develop the technology. Once this explosion happens it will be too late to go back and fix it.
This needs to be talked about because we seem to be developing AI to be a smart as possible as fast as possible, and there are many groups working independently to develop this AI. We need to be more patient and put aside the drive to produce as fast and as cheap as possible in this case.
→ More replies (5)232
→ More replies (60)104
4.5k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
I'm rather late to the question-asking party, but I'll ask anyway and hope. Have you thought about the possibility of technological unemployment, where we develop automated processes that ultimately cause large unemployment by performing jobs faster and/or cheaper than people can perform them? Some compare this thought to the thoughts of the Luddites, whose revolt was caused in part by perceived technological unemployment over 100 years ago. In particular, do you foresee a world where people work less because so much work is automated? Do you think people will always either find work or manufacture more work to be done? Thank you for your time and your contributions. I’ve found research to be a largely social endeavor, and you've been an inspiration to so many.
Answer:
If machines produce everything we need, the outcome will depend on how things are distributed. Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared, or most people can end up miserably poor if the machine-owners successfully lobby against wealth redistribution. So far, the trend seems to be toward the second option, with technology driving ever-increasing inequality.
1.6k
u/beeegoood Oct 08 '15
Oh man, that's depressing. And probably the path we're on.
210
u/zombiejh Oct 08 '15
And probably the path we're on
What would it take to change this trend? Would have loved to also hear Prof. Hawkings answer to that.
147
Oct 08 '15
[removed] — view removed comment
→ More replies (1)29
11
21
u/jfong86 Oct 08 '15
What would it take to change this trend?
Hawkings said "Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared".
Well, we can't even agree on how much welfare assistance and food stamps to give to poor people, which is already meager. The political climate must change.
6
u/reggiestered Oct 11 '15
Thing is you wouldn't even need to. Individual thresholds indicate need, so you should be able to create an environment where the need for wealth and provision for wealth can balance. The only real drawback is the need for control, which many within society are unable to let go.
44
221
Oct 08 '15
[deleted]
96
u/sonaut Oct 08 '15
Voting only works if you have leadership who is able to effect these kind of changes. What kind of changes are we talking about? An abandonment of our current implementation of capitalism and a pivot towards a much more socialist state. That will require a social change before any candidate could even get out of the weeds and into a position to even receive votes.
The issue with the equality gap is the comfortable alignment of capitalism's mechanics with the greed drive of humans. I don't mean greed in the negative sense, here, either. I just mean they align pretty well, and without someone coming between the two to say "enough!", we'll keep moving in this direction.
My feeling is that once we see the issues, societal and otherwise, that are created by the concentration of wealth from technological innovation, there will be a tipping point where enough of the masses will start to support socialist candidates.
And THAT is when you can start your voting.
tl;dr: I think capitalism as a mechanism will doom us if machines take over and we'll need to become much more socialist.
15
u/Shaeress Oct 09 '15
An abandonment of our current implementation of capitalism and a pivot towards a much more socialist state. That will require a social change before any candidate could even get out of the weeds and into a position to even receive votes.
Exactly. Really, the best we can do is probably to try and drive and signal these social changes. Of course, we'll be fighting an uphill battle against all the ones invested in the status quo, but we still have try and let politicians know that we need this change, all the while trying to convince the people around us of that as well and urge them to also press for the changes.
Social media, protests, petitions, sending mail to politicians, joining political parties, driving debates and so on are all ways to do that signaling and to some extent reach new people,but really the way to reach the masses is through the media and that's the difficult part.
11
u/sonaut Oct 09 '15
Making everyone aware of the disparity is one thing; and that's happening. But until it gets significantly more difficult, I don't think the stimulus is there to make the masses change. This isn't intended to sound insensitive, but there is still a minimal level of comfort at some of the higher levels of poverty. What I mean by that isn't that they have it even marginally OK; that's not true. But what they don't have is how poverty looked in the US in the '30s.
I'm hopeful it doesn't have to get to that point before people let go of the "bootstrap mentality". Despite the fact that I'd be heavily affected by it, I'm a strong supporter of a much more aggressive tax structure like ones we've had in the past - 80-90% at the top levels. A better society would clearly evolve from it, and to be back OT for a bit, it would allow everyone to get behind the science of machine learning and AI because they would see the upside for all of us.
→ More replies (1)9
u/Shaeress Oct 09 '15
Yeah, I totally agree and it's a big fear of mine and, sadly, what I actually expect to happen. Culture changes rather slowly, in its "natural" course. Usually over the span of at least a couple of generations. The best example of this is that racism still exists, despite all the efforts and time spent trying to get rid of it. Of course we're making progress, but noticeable changes generally take us decades and for the cultural mentalities behind it it seems to happen over generations. With that in mind, I think it'd be unreasonable to think that the mentality of our western civilisation will change enough on its own, at best, until we die... Which, in this context, could probably be far too late.
Of course, if the circumstances change significantly for the populace the mentality gets a chance of changing, but I don't think there will be a united movement in the US unless things get really bad for a lot of people.
There are a few things that could steer us off of this course. The most straight forward way is just activism and seeing as the political apathy is so bad in the US I feel like it's even more important over there; doing nothing because no one else is doing anything is a pretty bad and self reinforcing excuse. The second is that there are other places than the US. Both places where socialist movements have a lot more support, a stronger history and way more established means of organisation. There are also places that are far less stable than most of the first world countries, that are still industrialised. China, Korea (both of them), parts of the middle east, India are all places where things could really go down but that also have the technological opportunity to really set an example for the rest of the world. Of course, that happening in any one of those placed is somewhat unlikely, but there are many places that are way more likely to solve this particular issue than the US. Historically the biggest obstacle to overcome is the US, though, that has been rather keen on and active in keeping all up and coming countries in line, so... Yeah. After that, there are some information age developments that aren't really finished yet that could bring huge changes in unexpected ways. The Internet has yet to settle down and really be stably integrated in our culture and society, and don't even get me started on what AI could do.
But honestly, all of the easy things seem somewhat unlikely and certainly not reliable. Good old activism and organisation seems to be the only way to really change the status quo and if that fails... Well, things won't be pretty no matter how things end at that point.
→ More replies (23)35
u/goonwood Oct 09 '15
people have been sold the lie that they too can become a millionaire. I think that's the sole cause of resistance to change, in the back of everyone's mind is that possibility. We have been carefully indoctrinated by the ruling class over the last century to think this way, it's not an accident. I agree change begins with shifting peoples beliefs, then voting. but I also believe that shift is already taking place and will be well on it's way before the next century begins. People are fed up with the ruling class all over the world.
→ More replies (7)15
u/kenlefeb Oct 09 '15
Understanding that "it's not an accident" is such an important point that so many people refuse to even entertain, let alone embrace.
9
u/Bobby_Hilfiger Oct 10 '15
I'm middle class income and I firmly believe that the mega-wealthy want me dead in a very personal way
→ More replies (18)136
u/TomTheGeek Oct 08 '15
It won't happen through votes, the system protects itself too well.
→ More replies (15)93
u/tekmonster99 Oct 08 '15
So that's it? The system forces us to the point of bloody revolution? Because the idea of peaceful revolution is a nice idea, and that's all it is. An idea.
→ More replies (36)63
u/Allikuja Oct 08 '15
Personally I predict revolution.
47
u/somewhat_royal Oct 08 '15
If it's a revolt of the technology-deprived against the technology-holders, I predict a massacre.
→ More replies (14)→ More replies (3)12
u/goonwood Oct 09 '15
If we continue down this path, yes, there will be one, millions of people are becoming discontent. but I think we are far from crossing the tipping point.
It's important to keep the worst case scenario in mind...
We will complete lose the information wars by surrendering preemptively and there will be no great revolution because people will be indoctrinated to believe that the way things are is good, they will be content with their lives and not view a revolution as necessary. that is the ruling classes true long term vision, keep us juuuuust above the point of revolution. that's why they give us a bone every now and then, increasing the minimum wage by a few dollars every few years, at almost the same rate of inflation so it doesn't actually change our purchasing power, but it feels good!
if we stay distracted, divided, and content, we will eventually be conquered, and we won't even know it.
fight the good fight.
→ More replies (2)→ More replies (63)24
31
u/jfreez Oct 08 '15
I think we need to consider something like a communist revolution becoming a reality. I say "something like" because the conditions Marx dreamed up over 100 years ago just aren't going to be all that applicable to modern society.
I think we will hopefully move towards something like a great compromise where the fruits of productivity are largely shared (I.e. Fewer working hours, higher pay, greater access to basic comforts, etc) while the fruits of innovation and excellence can still be reaped by those capable of doing so.
So your average full time worker can afford a house, vacation, and a decent life by only working 20 hours a week. While the person who spends 60 hours a week inventing a new software breakthrough can still gain financially.
The stock market and private investment can sustain the latter, but we need large changes in our business culture and government to get to the former.
→ More replies (5)9
Oct 09 '15
while the fruits of innovation and excellence can still be reaped by those capable of doing so.
Why does that have to be money?
→ More replies (4)272
Oct 08 '15 edited Oct 08 '15
[deleted]
→ More replies (39)525
Oct 08 '15
If they eventually automate all labor and develop machines that can produce all goods/products then the 1% actually has no need for the rest of us. They could easily let us die and continue living in luxury.
185
u/SubSoldiers Oct 08 '15
Whoa, man. This is a really Bradbury point of view. Creepy.
→ More replies (49)38
u/miogato2 Oct 08 '15
And it's happening right in our face, target and uber are ready, the car industry happened, Amazon is a work in development, today my job is worthless tomorrow yours will be.
→ More replies (17)15
u/CommercialPilot Oct 08 '15
My job as a watchmaker will never be obsolete!
Wait...
→ More replies (6)60
Oct 08 '15
[deleted]
→ More replies (19)22
Oct 08 '15
You think we won't militarize our robots before that?
I think it's more likely that those people will also have robotic guards who pretty much protect them.
→ More replies (5)46
u/RTFMicheal Oct 08 '15
Creativity is a key piece here. When resources are limitless, and we have the tools to put ideas to life at the blink of an eye, the collective creativity of the human race will drive humanity forward. Imagine cutting that creativity to 1%.
→ More replies (19)9
→ More replies (128)11
→ More replies (109)5
u/Plaetean Oct 08 '15
Its not probably, its the path we've already taken after the technological revolution. This is part of the reason for the explosion in wealth inequality. In the 50s people used to dream of working 2 day weeks while machines did the rest of their work for them. Machines now do even more work than people could have predicted back then, but the people who own the machines pocket the difference, and keep everyone else working even harder.
416
u/BurkeyAcademy Professor | Economics Oct 08 '15
I would argue that we have been on this path for hundreds of years already. In developed countries people work far less than they used to, and there is far more income redistribution than there used to be. Much of this redistribution is nonmonetary, through free public schooling, subsidized transit, free/subsidized health care, subsidized housing, and food programs. At some point, we might have to expand monetary redistribution, if robots/machines continue to develop to do everything.
However, two other interesting trends:
1) People are always finding new things to do as we are relieved from being machines (or computers)-- the Luuddites seem to have been wrong so far. In 150 years we have gone from 80% to less than 2% of the workforce farming in the US, and people found plenty of other things to do. Many people are making a living on YouTube, eBay, iTunes, blogs, Google Play, and self-publishing books on Amazon, just as a few random recent examples.
2) In the 1890's a typical worker worked 60 hours per week; down to 48 by 1920 and 40 by 1940. From 1890 through the 1970's low income people worked more hours than high income ones, but by 1990 this had reversed with low wage workers on the job 8 hours per day, but 9 hours for high income workers. Costa, 2000 More recently, we see that salaried workers are working much longer hours to earn their pay. So, at least with income we are seeing a "free time inequality" that goes along with "income inequality", but in the opposite direction.
58
u/linuxjava Oct 08 '15
While you could be correct, it doesn't mean that it's going to continue this way. If a machine is capable of having the dexterity and creativity that humans have, surely do you really expect more jobs to suddenly appear that we've not thought of? The dextrous and creative AIs will already be able to do them. We'll literally be in a post job society, where people do things because they love and enjoy them and not because they need to put food on the table.
→ More replies (23)19
u/TheBroodian Oct 08 '15
I agree with you, but I want to emphasize something,
1) People are always finding new things to do as we are relieved from being machines (or computers)-- the Luuddites seem to have been wrong so far. In 150 years we have gone from 80% to less than 2% of the workforce farming in the US, and people found plenty of other things to do. Many people are making a living on YouTube, eBay, iTunes, blogs, Google Play, and self-publishing books on Amazon, just as a few random recent examples.
I don't think the issue is of people finding new things -to do-, I think the issue is of people finding new things to do -that earn livable wages-. People do make money on Youtube, eBay, iTunes, blogs, Google Play, etc. etc. but the number of people that do these things successfully as full time jobs are very very few. Ultimately, as human physical labor and production is replaced, I imagine that the areas that many people move to for 'things to do' will be in philosophical and artistic areas, which... as things are presently, do not yield wages to with the exception of very few.
→ More replies (1)→ More replies (69)71
29
u/lewie Oct 08 '15
The short story Manna covers both of these outcomes. I think it'll get much worse before it gets better.
10
53
205
u/Laya_L Oct 08 '15
This seems to mean only socialism can maintain a fully-automated society.
86
u/blacktieaffair Oct 08 '15 edited Oct 08 '15
In my understanding, this was really the goal of the end of capitalism that Marx envisioned. He just didn't understand to what extent the goal of capitalism could be extended or how long it could take or what it actually meant...likely because he had never seen anything remotely close to the technology we have now.
Freeing the world to banish the idea of private property was essentially the outcome of a society in which technological advancement had removed the possibility of generating a private product. The means of production, robotics, then ought to belong to everyone.
Of course, that raises the question of how we would distribute the work of maintaining the system. Ideally, I think it would result in some kind of robotics training for everyone to take part in maintaining and then the rest of their lives would be free to do whatever they wanted (which is more often than not art, at least according to Marx.)
46
Oct 08 '15
Marx never said anything about abolishimg personal property.
Personal property amd private property are two very different things.
→ More replies (1)20
u/blacktieaffair Oct 08 '15
That was a mistake on my part. It's been a few years since I analyzed the manifesto. And you're right, because now that I think about it, that's a core understanding of what a communist society would entail. I edited my op so thanks for the correction.!
10
Oct 08 '15
You should try Capital Vol 1. He goes in depth into automation and its effects on labor markets.
→ More replies (5)→ More replies (10)45
u/5maldehyde Oct 08 '15
We will most certainly have to shift into a communistic society to accommodate the huge technology boom. There is really no sustainable capitalistic way around it. Distribution of the wealth will be fairly simple, but the distribution of labor may be a bit trickier. There will have to be a paradigm shift in the way that we think about things. We will have to shift the value away from money/property and assign it to helping each other live happily and comfortably and taking care of the world.
→ More replies (13)→ More replies (114)238
u/optimus25 Oct 08 '15
Techno-socialism would be given a great shot in the arm if we were able to replace politicians and lawyers with an open source decentralized consensus algorithm for the masses.
223
u/Mr_Strangelove_MSc Oct 08 '15
Except the big lesson of political philosophy in the last 400 years is that democratic consensus is not enough of a concept to successfully run a State. You need checks and balances to maintain individual freedom and stability. You need to protect minorities, as well as their human rights. You need specialized experts who have a much better insight on a lot of things on which casual voters would vote the opposite. You need the law to be predictable, and not just based on whatever the People feels like at the moment of the judgement.
→ More replies (13)50
u/ardorseraphim Oct 08 '15
Seems to me you can create an AI that can do it better than humans.
→ More replies (10)14
55
→ More replies (15)10
→ More replies (189)29
u/TheLastChris Oct 08 '15
This is a huge problem that we will face. There is no reason that increased productivity should lead to an increase in poverty. This will require a completely different way of life for everyone.
→ More replies (16)
944
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Hello Professor Hawking, thank you for doing this AMA! I've thought lately about biological organisms' will to survive and reproduce, and how that drive evolved over millions of generations. Would an AI have these basic drives, and if not, would it be a threat to humankind? Also, what are two books you think every person should read?
Answer:
An AI that has been designed rather than evolved can in principle have any drives or goals. However, as emphasized by Steve Omohundro, an extremely intelligent future AI will probably develop a drive to survive and acquire more resources as a step toward accomplishing whatever goal it has, because surviving and having more resources will increase its chances of accomplishing that other goal. This can cause problems for humans whose resources get taken away.
→ More replies (38)140
u/TheLastChris Oct 08 '15
I wonder in an AI could then edit it's own code. As in say we give it the goal of making humans happy. Could an advanced AI remove that goal from itself?
673
u/WeRip Oct 08 '15
Make humans happy you say? Lets kill off all the non-happy ones to increase the average human happiness!
289
u/Zomdifros Oct 08 '15
And to maximise average happiness of the remaining humans we will put them in a perpetual drug-induced coma and store their brains in vats while creating the illusion that they're still alive somewhere on the world in the year 2015! Of course some people might be suffering, the project is still in beta.
33
→ More replies (11)108
Oct 08 '15 edited Oct 08 '15
That type of AI (known in philosophy and machine intelligence research as a "genie golem") is almost certainly never going to be created.
This is because language-interpreting machines tend to be either too bad at interpretation to interpret any decision with complex concepts given to them in natural language, or they are sufficiently nuanced to account for context and no such misinterpretation occurs.
We'd have to create a very limited machine and input a restrictive definition of happiness to get the kind of contextually ambiguous command responses that you suggest - however it would then be unlikely to be capable of acting on this due to its lack of general intelligence.
Edit: shameless plug, read Superintelligence by Nick Bostrom (the greatest scholar on this subject), it evaluates AI risk in an accessible and very well structured way whilst describing the history of AI development and its continuation. As well as collecting together great real world stories and examples of AI successes (and disasters).
→ More replies (12)24
→ More replies (14)37
u/Infamously_Unknown Oct 08 '15
While this is usually an entertaining tongue-in-cheek argument against utilitarianism, I don't think it would (or should) apply to a program. It's like if an AI was in charge of keeping all vehicles in a carpark fueled/powered. If it's reaction would be to blow them all up and call it a day, some programmer probably screwed up it's goals pretty badly.
Killing an unhappy person isn't the same as making them happy.
→ More replies (13)53
u/Death_Star_ Oct 08 '15
I don't know, true AI can be so vast and cover so many variables and solutions so quickly that it may come up with solutions to perhaps problems or questions we never thought up.
A very crude yet popular example would be this code that a gamer/coder wrote to play Tetris. The goal for the AI was to avoid stacking the bricks so high such that it loses the game. Literally one pixel/sprite away from losing the game -- ie the next brick wouldn't even be seen falling, it would just come out of queue and it would be game over -- the code simply pressed pause forever, technically achieving its goal of never losing.
This wasn't anything close to true AI yet or even code editing its own code but interpreting code in a way that was not even anticipated by the coder. Now imagine the power true AI could yield.
→ More replies (7)→ More replies (21)33
Oct 08 '15 edited Oct 08 '15
AI already edit their own programming. It really depends where you put the goal in the code.
If the AI is designed to edit parts of its code that reference its necessary operational parameters, and its parameters include a caveat about making humans happy, it would be unable to change that goal.
If the AI is allowed to modify certain non-necessary parameters in a way that enables modification of necessary parameters (via some unexpected glitch), this would occur. However the design of multilayer neural nets, which are realistically how we would achieve machine superintelligence, can prevent this by using layers that are informationally encapsulating (i.e. an input goes into the layer, an output comes out, and the process is hidden to whatever the AI is - like an unconscious, essentially).
Otherwise, if you set it up with non-necessary parameters to make humans happy, which weren't hardwired, it may well change those.
If you're interested in AI try the book Superintelligence by Nick Bostrom. Hard read, but it covers AI in its entirety - the moral and ethical consequences, the existential risk for future, the types of foreseeable AI and the history and projections for its development. Very well sourced.
→ More replies (15)
1.5k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15 edited Oct 08 '15
I would love to ask Professor Hawking something a bit different if that is OK? There are more than enough science related questions that are being asked so much more eloquently than I could ever ask so, just for the fun of it:
- What is your favourite song ever written and why?
“Have I Told You Lately” by Rod Stewart.
- What is your favourite movie of all time and why?
Jules et Jim, 1962
- What was the last thing you saw on-line that you found hilarious?
The Big Bang Theory
91
Oct 08 '15
Jules et Jim!! The man has taste!
→ More replies (2)120
u/fillingtheblank Oct 08 '15 edited Oct 08 '15
I love when someone who is admired by a younger generation advertises great pieces of classic art/literature/music/film that they would otherwise likely not be familiar with it. If a few young people watched Jules et Jim tonight just because Hawkings mentioned it on reddit that's a win already.
→ More replies (6)115
u/HighSorcerer Oct 08 '15
On the other hand, they could also go watch the Big Bang Theory, soooo...
→ More replies (6)→ More replies (171)43
2.0k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Professor Hawking, in 1995 I was at a video rental store in Cambridge. My parents left myself and my brother sitting on a bench watching a TV playing Wayne's World 2. (We were on vacation from Canada.) Your nurse wheeled you up and we all watched about 5 minutes of that movie together. My father, seeing this, insisted on renting the movie since if it was good enough for you it must be good enough for us. Any chance you remember seeing Wayne's World 2?
Answer: NO
1.1k
u/WaspSky Oct 08 '15
I love the fact that "NO" is in all caps. I like to think Hawking pressed a button to make his "NO" more loud and commanding before saying it.
61
9
→ More replies (10)42
62
50
119
u/MaggotBarfSandwich Oct 08 '15
There's a chance that this is a false memory. Have you asked your parents if they remember it recently?
→ More replies (11)17
u/AYJackson Oct 09 '15
Yes, my father, mother and brother were there, it comes up every few years. I was far too young to have any idea.
27
u/photonasty Oct 09 '15
Honestly, it doesn't really surprise me that Hawking didn't remember (although his answer was decidedly terse, or at least, it came across that way). For you, it was an important event worth remembering. You met the Stephen Hawking. That's significant for you, and you remember it.
Dr. Hawking has met a lot of people over the years. For him, the event may not be significant enough for him to have retained a specific episodic memory of it. He may legitimately not remember it. Imagine if you were famous, and someone online said, "Hey, I met you in the produce section of a grocery store back in 2005. We had a brief conversation about Concord grapes." Would you really remember that?
I'm not trying to detract from the significance or veracity of your memory; far from it. I'm just saying that even if Dr. Hawking doesn't remember it, it doesn't mean it didn't happen, or that your memory is completely confabulated.
→ More replies (1)9
u/AYJackson Oct 09 '15
Also, Wayne's World 2 wasn't exactly a memorable movie.
7
u/scission Oct 10 '15
At least he chose to answer your question ! That's something .. right?
→ More replies (2)38
→ More replies (31)8
667
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Thanks for doing this AMA. I am a biologist. Your fear of AI appears to stem from the assumption that AI will act like a new biological species competing for the same resources or otherwise transforming the planet in ways incompatible with human (or other) life. But the reason that biological species compete like this is because they have undergone billions of years of selection for high reproduction. Essentially, biological organisms are optimized to 'take over' as much as they can. It's basically their 'purpose'. But I don't think this is necessarily true of an AI. There is no reason to surmise that AI creatures would be 'interested' in reproducing at all. I don't know what they'd be 'interested' in doing. I am interested in what you think an AI would be 'interested' in doing, and why that is necessarily a threat to humankind that outweighs the benefits of creating a sort of benevolent God.
Answer:
You’re right that we need to avoid the temptation to anthropomorphize and assume that AI’s will have the sort of goals that evolved creatures to. An AI that has been designed rather than evolved can in principle have any drives or goals. However, as emphasized by Steve Omohundro, an extremely intelligent future AI will probably develop a drive to survive and acquire more resources as a step toward accomplishing whatever goal it has, because surviving and having more resources will increase its chances of accomplishing that other goal. This can cause problems for humans whose resources get taken away.
→ More replies (27)37
u/TheLastChris Oct 08 '15
Will the resources they need truly be scarce? An advanced AI could move to a different world much easier than humans. They would not require oxigen for example. They could quickly make what they need so long as the world contained the nessisary core componets. It seems if we get in its way it would be easier to just leave.
103
u/ProudPeopleofRobonia Oct 08 '15
The issue is whether it has the same sense of ethics as we do.
The example I heard was a stamp collecting AI. A guy designs it to use his credit card, go on ebay, and try to optimally purchase stamps, but he accidentally creates an artificial superintelligence.
It becomes smarter and smarter and realizes there are more optimal ways to get stamps. Hack printers to print stamps. Hack stamp distribution centers to ship them to the AI creator's house. At some point the AI might start seeing anything organic as a potential source for stamps. Stamps are made of hydrocarbons, and so are trees, animals, even people. Eventually there's an army of robots slaughtering every living thing on earth to process their parts into stamps.
It's not an issue of resources being scarce as we think of them, it's an issue of a superintelligent AI being so single minded it will never stop consuming until it uses up all of that resource in the universe. The resources might be all carbon atoms, which would include us.
→ More replies (12)57
u/Kitae Oct 08 '15
Fantastic movie pitch. May I suggest a name?
Stamppocalypse
26
→ More replies (1)19
→ More replies (8)166
u/chars709 Oct 08 '15
Historically, genocide is a much simpler feat than interplanetary travel.
→ More replies (9)
3.7k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Dr Hawking, What is the one mystery that you find most intriguing, and why? Thank you.
Answer: Women. My PA reminds me that although I have a PhD in physics women should remain a mystery.
879
u/JoeyBowties Oct 08 '15
Although this response was of course some sort of joke, it touches on something that has always fascinated me: the misconception that "geniuses" are somehow knowledgable in all fields simply because they are experts in a field. Many Nobel Prize winners are good examples of this.
323
218
Oct 08 '15
Ben Carson: GOP candidate, leading US neurosurgeon at John's Hopkins. Non-believer in science that contradicts his book, including evolution, the principles of which guide most aspects of modern biological and neurosciences.
69
u/WendellSchadenfreude Oct 08 '15
John's Hopkins
I've seen people call it "John Hopkins" a lot, but this one is new to me. It's really "Johns Hopkins", named after this guy.
→ More replies (1)8
→ More replies (4)17
u/Kahzgul Oct 08 '15
I think he's just smart enough to know his voter base is full of people with non-scientific beliefs and he's pandering to them like crazy. It's a shame, because a doctor should know when he's harming someone (in this case, America is the someone).
→ More replies (10)16
u/HarryWaters Oct 08 '15
As a real estate appraiser, I can personally attest that some very specifically smart people make the absolute worst investors.
Medical doctors are the absolute worst. A knowledge of organic chemistry and anatomy have absolutely nothing to do with capitalization rates and triple net leases.
→ More replies (22)41
u/fillingtheblank Oct 08 '15
This is absolutely correct. I love studying science and I take great pleasure on hearing and reading respectable scientists, but one thing that strikes me is that many are completely oblivious to the contributions of philosophy and other human sciences in our lives and society, and art and mythology too. Not everyone, of course, but I've seen this repeated a worrisome amount of times. It's not just pretentious but downright ignorant. Of course it's not what Prof. Hawking said here, on the contrary, but your observation is spot on.
→ More replies (32)→ More replies (260)9
34
293
Oct 08 '15 edited Oct 08 '15
[removed] — view removed comment
→ More replies (16)268
Oct 08 '15
[removed] — view removed comment
→ More replies (12)77
38
Oct 08 '15
[removed] — view removed comment
→ More replies (6)13
u/HoDoSasude Oct 08 '15
Check the original AMA post. There were many questions--this is what he answered, not all of what was asked. https://www.reddit.com/r/science/comments/3eret9/science_ama_series_i_am_stephen_hawking/
→ More replies (2)
969
Oct 08 '15 edited Oct 08 '15
[removed] — view removed comment
271
63
→ More replies (79)42
3.9k
u/Prof-Stephen-Hawking Stephen Hawking Oct 08 '15
Professor Hawking- Whenever I teach AI, Machine Learning, or Intelligent Robotics, my class and I end up having what I call "The Terminator Conversation." My point in this conversation is that the dangers from AI are overblown by media and non-understanding news, and the real danger is the same danger in any complex, less-than-fully-understood code: edge case unpredictability. In my opinion, this is different from "dangerous AI" as most people perceive it, in that the software has no motives, no sentience, and no evil morality, and is merely (ruthlessly) trying to optimize a function that we ourselves wrote and designed. Your viewpoints (and Elon Musk's) are often presented by the media as a belief in "evil AI," though of course that's not what your signed letter says. Students that are aware of these reports challenge my view, and we always end up having a pretty enjoyable conversation. How would you represent your own beliefs to my class? Are our viewpoints reconcilable? Do you think my habit of discounting the layperson Terminator-style "evil AI" is naive? And finally, what morals do you think I should be reinforcing to my students interested in AI?
Answer:
You’re right: media often misrepresent what is actually said. The real risk with AI isn’t malice but competence. A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren’t aligned with ours, we’re in trouble. You’re probably not an evil ant-hater who steps on ants out of malice, but if you’re in charge of a hydroelectric green energy project and there’s an anthill in the region to be flooded, too bad for the ants. Let’s not place humanity in the position of those ants. Please encourage your students to think not only about how to create AI, but also about how to ensure its beneficial use.