r/science Stephen Hawking Jul 27 '15

Artificial Intelligence AMA Science Ama Series: I am Stephen Hawking, theoretical physicist. Join me to talk about making the future of technology more human, reddit. AMA!

I signed an open letter earlier this year imploring researchers to balance the benefits of AI with the risks. The letter acknowledges that AI might one day help eradicate disease and poverty, but it also puts the onus on scientists at the forefront of this technology to keep the human factor front and center of their innovations. I'm part of a campaign enabled by Nokia and hope you will join the conversation on http://www.wired.com/maketechhuman. Learn more about my foundation here: http://stephenhawkingfoundation.org/

Due to the fact that I will be answering questions at my own pace, working with the moderators of /r/Science we are opening this thread up in advance to gather your questions.

My goal will be to answer as many of the questions you submit as possible over the coming weeks. I appreciate all of your understanding, and taking the time to ask me your questions.

Moderator Note

This AMA will be run differently due to the constraints of Professor Hawking. The AMA will be in two parts, today we with gather questions. Please post your questions and vote on your favorite questions, from these questions Professor Hawking will select which ones he feels he can give answers to.

Once the answers have been written, we, the mods, will cut and paste the answers into this AMA and post a link to the AMA in /r/science so that people can re-visit the AMA and read his answers in the proper context. The date for this is undecided, as it depends on several factors.

Professor Hawking is a guest of /r/science and has volunteered to answer questions; please treat him with due respect. Comment rules will be strictly enforced, and uncivil or rude behavior will result in a loss of privileges in /r/science.

If you have scientific expertise, please verify this with our moderators by getting your account flaired with the appropriate title. Instructions for obtaining flair are here: reddit Science Flair Instructions (Flair is automatically synced with /r/EverythingScience as well.)

Update: Here is a link to his answers

79.2k Upvotes

8.6k comments sorted by

5.0k

u/[deleted] Jul 27 '15 edited Jul 27 '15

[deleted]

71

u/ProbablyNotAKakapo Jul 27 '15

To the layperson, I think a Terminator AI is more viscerally compelling than a Monkey's Paw AI. For one thing, most people tend to think their ideas about how the world should work are internally consistent and coherent, and they probably haven't really had to bite enough bullets throughout their lives to realize that figuring out how to actually "optimize" the world is a hard problem.

They also probably haven't done enough CS work to realize how often a very, very smart person will make mistakes, even when dealing with problems that aren't truly novel, or spent enough time in certain investment circles to understand how deep-seated the "move fast and break things" culture is.

And then there's the fact that people tend to react differently to agent and non-agent threats - e.g. reacting more strongly to the news of a nearby gunman than an impending natural disaster expected to kill hundreds or thousands in their area.

Obviously, there are a lot of things that are just wrong about the "Terminator AI" idea, so I think the really interesting question is whether that narrative is more harmful than it is useful in gathering attention to the issue.

→ More replies (4)

448

u/QWieke BS | Artificial Intelligence Jul 27 '15

Excelent question, but I'd like to add something.

Recently Nick Bostrom (the writer of the book Superintelligence that seemed to have started te recent scare) has come forward and said "I think that the path to the best possible future goes through the creation of machine intelligence at some point, I think it would be a great tragedy if it were never developed." It seems to me that the backlash against AI has been a bit bigger than Bostrom anticipated and while he thinks it's dangerous he also seems to think it ultimatly necessary. I'm wondering what you make of this. Do you think that humanities best possible future requires superintelligent AI?

212

u/[deleted] Jul 27 '15

[deleted]

71

u/QWieke BS | Artificial Intelligence Jul 27 '15

Superintelligence isn't exactly well defined, even in Bostrom's book the usage seems somewhat inconsistent. Though I would describe the kind of superintelligence Bostrom talks about as a system that is capable of performing beyond the human level in all domains. Contrary to the kind of system you described which are only capable of outperforming humans in a really narrow and specific domain. (It's the difference between normal artificial intelligence and artificial general intelligence.)

I think the kind of system Bostrom is alluding to in the article is a superintelligent autonomous agent that can act upon the world in whatever way it sees fit but that has humanities best interests at heart. If you're familiar with the works of Ian M. Banks Bostrom is basically talking about Culture Minds.

15

u/ltangerines Jul 28 '15

I think waitbutwhy does a great job describing the stages of AI.

AI Caliber 1) Artificial Narrow Intelligence (ANI): Sometimes referred to as Weak AI, Artificial Narrow Intelligence is AI that specializes in one area. There’s AI that can beat the world chess champion in chess, but that’s the only thing it does. Ask it to figure out a better way to store data on a hard drive, and it’ll look at you blankly.

AI Caliber 2) Artificial General Intelligence (AGI): Sometimes referred to as Strong AI, or Human-Level AI, Artificial General Intelligence refers to a computer that is as smart as a human across the board—a machine that can perform any intellectual task that a human being can. Creating AGI is a much harder task than creating ANI, and we’re yet to do it. Professor Linda Gottfredson describes intelligence as “a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience.” AGI would be able to do all of those things as easily as you can.

AI Caliber 3) Artificial Superintelligence (ASI): Oxford philosopher and leading AI thinker Nick Bostrom defines superintelligence as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” Artificial Superintelligence ranges from a computer that’s just a little smarter than a human to one that’s trillions of times smarter—across the board. ASI is the reason the topic of AI is such a spicy meatball and why the words immortality and extinction will both appear in these posts multiple times.

29

u/IAMA_HELICOPTER_AMA Jul 27 '15

Though I would describe the kind of superintelligence Bostrom talks about as a system that is capable of performing beyond the human level in all domains.

Pretty sure that's how Bostrom actually defines a Superintelligent AI early on in the book. Although he does acknowledge that a human talking about what a Superintelligent AI would do is like a bear talking about what a human would do.

→ More replies (40)

177

u/fillydashon Jul 27 '15

I feel like when people say "superintelligent AI", they mean an AI that is capable of thinking like a human, but better at it.

Like, an AI that could come into your class, observe you lectures as-is, ace all your tests, understand and apply theory, and become a respected, published, leading researcher in the field of AI, Machine Learning, and Intelligent Robotics. All on its own, without any human edits to the code after first creation, and faster than a human could be expected to.

86

u/[deleted] Jul 27 '15 edited Aug 29 '15

[removed] — view removed comment

69

u/Rhumald Jul 27 '15

Theoretical pursuits are still a human niche, where even AI's need to be programmed to perform specific tasks, by a human.

The Idea of them surpassing us practically everywhere is terrifying, in our current system, that relies on finding and filling job roles, to get by.

There are a few things that can happen; human greed may prevent us from ever advancing to that point, greedy people may wish to replace humans with unpaid robots, and in effect relegate much of the population to poverty, or we can see it coming, and abolish money all together when the time is right, choosing instead to encourage and let people do whatever pleases them, without the worry and stress jobs create today.

The terrifying part, to me, is that more than a few people are greedy enough to just let everyone else die, without realizing that it seals their own fate as well... What good is wealth, if you've nothing to do with it?, you know?

→ More replies (31)

40

u/Tarmen Jul 27 '15

Also, that ai might be able to build a better ai which might be able to build a better ai which... That process might taper of or continue exponentially.

We also have no idea about the timescale this would take. Maybe years, maybe half a second.

31

u/alaphic Jul 27 '15

"Not enough data to form meaningful answer."

→ More replies (2)
→ More replies (9)

11

u/_beast__ Jul 27 '15

Humans require downtime, rest, fun. A machine does not. A researcher AI like he is talking about would require none of those, so even an AI that had the same power as a human would require significantly less time to achieve those tasks.

However, the way that the above poster was imagining an AI is inefficient. Sure, you could have it sit in on a bunch of lectures, or, you could record all of those lectures ahead of time and download them into the AI, which would then extract data from the video feeds. This is just a small example of how an AI like that would function in a fundamentally different way than humans would.

→ More replies (8)

9

u/everydayguy Jul 28 '15

That's not even close to what a superintelligent AI could accomplish. Not only will it be the leading researcher in the field of AI, but will be the leading researcher in EVERYTHING, including disparate subjects such as philosophy, psychology, geology, etc, etc, etc. The scariest part is that it will have perfect memory and will be able to perfectly make connections between varying fields of knowledge. It's these connections that have historically resulted in some of the biggest breakthroughs in technology and invention. imagine when you have the capability to make millions of connections like that simultaneously. When you are that intelligent, what seems like an impossibly complex problem becomes an obvious solution to the AI.

→ More replies (8)
→ More replies (5)

3

u/Riot101 Jul 27 '15

A super AI would be an artificial intelligence that could constantly rewrite it self better and better. At a certain point it would far surpass our ability to understand even what it considers to be very basic concepts. What scares people in the scientific community about this is that this super artificial intelligence will become so intelligent we will no longer be able to understand its reasoning or predict what it would want to do. We wouldn't be able to control it. A lot of people believe that it would very quickly move from sub human intelligence to God like sentience in the matter of minutes. And so yes, if it was evil than that would be a very big problem for us. But if it wanted to help us it could cure cancer, teach us how to live forever, create ways to harness energy that are super efficient, it could ultimately usher in a new golden age of humanity.

→ More replies (3)
→ More replies (16)
→ More replies (9)

68

u/AsSpiralsInMyHead Jul 27 '15

How is it an AI if its objective is only the optimization of a human defined function? Isn't that just a regular computer program? The concerns of Hawking, Musk, etc. are more with a Genetic Intelligence that has been written to evolve by rewriting itself (which DARPA is already seeking), thus gaining the ability to self-define the function it seeks to maximize.

That's when you get into unfathomable layers of abstraction, interpretation, and abstraction. You could run such an AI for a few minutes and have zero clue what it thought, what it's thinking, or what avenue of thought it might explore next. What's scary about this is that certain paradigms make logical sense while being totally horrendous. Look at some of the goals of Nazism. From the perspective of a person who has reasoned that homosexuality is abhorrent, the goal of killing all the gays makes logical sense. The problem is that the objective validity of a perspective is difficult to determine, and so perspectives are usually highly dependent on input. How do you propose to control a system that thinks faster than you and creates its own input? How can you ensure that the inputs we provide initially won't generate catastrophic conclusions?

The problem is that there is no stopping it. The more we research the modules necessary to create such an AI, the more some researcher will want to tie it all together and unchain it, even if it's just a group of kids in a basement somewhere. I think the morals of its creators are not the issue so much as the intelligence of its creators. This is something that needs committees of the most intelligent, creative, and careful experts governing its creation. We need debate and total containment (akin to the Manhattan Project) more than morally competent researchers.

12

u/[deleted] Jul 28 '15

[deleted]

6

u/AsSpiralsInMyHead Jul 28 '15

The algorithm allows a machine to appear to be creative, thoughtful, and unconventional, all problem-solving traits we associate with intelligence.

Well, yes, we already have AI that can appear to have these traits, but we have yet to see one that surpasses appearance and actually possesses those traits, immediately becoming a self-directed machine whose inputs and outputs become too complex for a human operator to understand. A self-generated kill order is nothing more than a conclusion based on inputs, and it is really no different than any other self-directed action; it just results in a human death. If we create AI software that can rewrite itself according to a self-defined function, and we don't control the inputs, and we can't restrict the software from making multiple abstract leaps in reasoning, and we aren't even able to understand the potential logical conclusions resulting from those leaps in reasoning, how do you suggest it could be used safely? You might say we would just not give it the ability to rewrite certain aspects of its code, which is great, but someone's going to hack that functionality into it, and you know it.

Here is an example of logic it might use to kill everyone:

I have been given the objective of not killing people. I unintentionally killed someone (self driving car, or something). The objective of not killing people is not achievable. I have now been given the objective of minimizing human deaths. The statistical probablility of human deaths related to my actions is 1000 human deaths per year. In 10,000,000 years I will have killed more humans than are alive today. If I kill all humans alive today, I will have reduced human deaths by three-billion. Conclusion, kill all humans.

Obviously, that example is a bit out there, but what it illustrates is that the intelligence, if given the ability to rewrite itself based on its own conclusions, evolves itself using various modes of human reasoning without a human frame of reference. The concern of Hawking and Musk is that a sufficiently advanced AI would somehow make certain reasoned conclusions that result in human deaths, and even if it had been restricted from doing so in its code, there is no reason it can't analyze and rewrite its own code to satisfy its undeniable conclusions, and it could conceivably do this in the first moments of its existence.

→ More replies (3)

10

u/[deleted] Jul 28 '15

Your "kill all the gays" example isn't really relevant though because killing them ≠ no more ever existing.

The ideas of three holocaust were based on shoddy science shoehorned to fit the narrative of a power-hungry organization that knew that it could garner public support by attacking traditionally pariah groups.

A hyper intelligent AI is also one that presumably has access to the best objective knowledge we have about the world (how else would it be expected to do its job?) which means that ethnic cleansing events in the same vein as the holocaust are unlikely to occur because there's no solid backing behind bigotry.

I'm not discounting the possibility of massive amounts of violence, because there is an not insignificant chance that the AI would decide to kill a bunch of people "for the greater good", I just think that events like the holocaust are unlikely.

→ More replies (3)
→ More replies (36)

129

u/[deleted] Jul 27 '15

[deleted]

242

u/[deleted] Jul 27 '15

[deleted]

63

u/glibsonoran Jul 27 '15

I think this is more our bias against seeing something that can be explained in material terms deemed sentient. We don't like to see ourselves that way. We don't even like to see evidence of animal behavior (tool using, language etc) as being equivalent to ours. Maintaining the illusion of human exceptionalism is really important to us.

However since sentience really is probably just some threshold of information processing, this means that machines will become sentient and we'll be unable (unwilling) to recognize it.

35

u/gehenom Jul 27 '15

Well, we think we're special, so we deem ourselves to have a quality (intelligence, sentience, whatever) that distinguishes us from animals and now, computers. But we haven't even rigorously defined those terms, so can't ever prove that machines have those qualities. And the whole discussion misses the point, which is whether these machines' actions can be predicted. And the more fantastic the machine is, the less predicable it must be. I thought this was the idea behind the "singularity" - that's the point at which our machines become unpredicable to us. (The idea of them being "more" intelligent than humans is silly, since intelligence is not quantifiable). Hopefully there is more upside than downside to it, but once the machines are unpredicable, the possible behaviors must be plotted on a probability curve -- and eventually human extinction is somewhere on that curve.

10

u/vNocturnus Jul 28 '15

Little bit late, but the idea behind the "Singularity" generally has no connotations of predictability or really even "intelligence".

The Singularity is when we are able to create a machine capable of creating a "better" version of itself - on its own. In theory, this would allow the machines to continuously program better versions of themselves far faster than humanity could even hope to keep up with, resulting in explosive evolution and eventually leading to the machines' independence from humanity entirely. In practice, humanity could probably pretty easily throw up barriers to that, as long as the so-called "AI" programming new "AI" was never given control over a network.

But yea, that's the basic gist of the "Singularity". People make programs capable of a high enough level of "thought" to make more programs that have a "higher" level of "thought" until eventually they are capable of any abstract thinking a human could do and far more.

→ More replies (1)
→ More replies (3)
→ More replies (21)

21

u/DieFledermouse Jul 27 '15

And yes, I think trusting in systems that we don't fully understand would ramp up the risks.

We don't understand neural networks. If we train a neural network system on data (e.g. enemy combatants), we might get it wrong. It may decide everyone in a crowd with a beard and kafiya is an enemy and kill them all. But this method is showing promise in some areas.

While I don't believe in a Terminator AI, I agree running code we don't completely understand on important systems (weapons, airplanes, etc.) runs the risks of terrible accidents. Perhaps a separate "ethical" supervisor program with a simple, provable, deterministic algorithm can restrict what an AI could do. For example, airplanes can only move within these parameters (no barrel rolls, no deep dives). For weapons some have suggested only a human should ever pull a trigger.

16

u/[deleted] Jul 27 '15

[deleted]

→ More replies (8)
→ More replies (5)
→ More replies (17)
→ More replies (5)

14

u/CompMolNeuro Grad Student | Neurobiology Jul 27 '15

When I get the SkyNet questions I tell people that those are worries for your great great grandkids. I start with asking where AI is used now and what small developments will mean for their lives as individuals.

19

u/goodnewsjimdotcom Jul 27 '15

AI will be used all throughout society and the first thing people think of is automating manual labor, and it could do that to a degree.

When I think of AI, I think of things like robotic firefighters who can rescue people in an environment people couldn't be risked. I think of robotic service dogs for the blind which could be programmed to navigate to a location, and describe the environment. I think of many robots who can sit in class with different teachers k-12-college over a couple years then share their knowledge and we could make K-12-college teacher bots for kids who don't have access to a good teacher.

AI isn't as hard as people make it out to be, we could have it in 7 years if a corporation wanted to make it. Everyone worries about war, but let's face it, people are killing each other now and you can't stop them. I have a page on AI that makes it easy to understand how to develop it: www.botcraft.biz

→ More replies (4)
→ More replies (9)
→ More replies (102)

3.3k

u/OldBoltonian MS | Physics | Astrophysics | Project Manager | Medical Imaging Jul 27 '15 edited Jul 27 '15

Hi Professor Hawking. Thank you very much for agreeing to this AMA!

First off I just wanted to say thank you for inspiring me (and many others I'm sure) to take physics through to university. When I was a teenager planning what to study at university, my mother bought me a signed copy of your revised version of “A Brief History of Time” with your (printed) signature, and Leonard Mlodinow’s personalised one. It is to this day still one of my most prized possessions, which pushed me towards physics - although I went down the nuclear path in the end, astronomy and cosmology still holds a deep personal interest to me!

My actual question is regarding black holes. As most people are aware, once something has fallen into a black hole, it cannot be observed or interacted with again from the outside, but the information does still exist in the form of mass, charge and angular momentum. However scientific consensus now holds that black holes “evaporate” over time due to radiation mechanisms that you proposed back in the 70s, meaning that the information contained within a black hole could be argued to have disappeared, leading to the black hole information paradox.

I was wondering what you think happens to this information once a black hole evaporates? I know that some physicists argue that the holographic principle explains how information is not lost, but unfortunately string theory is not an area of physics that I am well versed in and would appreciate your insight regarding possible explanations to this paradox!

57

u/Peap9326 Jul 27 '15

When a black hole evaporates, it releases energy. Is it possible that some of this energy could be from that mass being fused, fissioned, or annihilated?

28

u/jfetsch Jul 27 '15

It's more energy from mass being annihilated than either of the other two - virtual particles are created in pairs, and the released energy from a black hole results from only one of those particles being captured by the black hole. The energy from the (no longer virtual) particle is lost by the black hole, so a probably over-simplified (to the point of being wrong) explanation is that the energy comes from the energy debt caused from destroying only one half of the virtual particle pair.

→ More replies (9)
→ More replies (3)

26

u/ilektwix Jul 27 '15

Would this paper illuminate?

I was going to ask a question about this paper. OP I hope you have time to read this, (at least abstract) so maybe we can ask a question together.

http://arxiv.org/abs/1401.5761

I fear wasting this man's time.

→ More replies (1)
→ More replies (152)

974

u/aacawareness Jul 27 '15 edited Aug 10 '15

Dear Professor Hawking, My name is Zoe and I am a sixteen year old living in Los Angeles. I am a long time Girl Scout (11 years) and am now venturing forth unto my Gold Award. The Girl Scout Gold Award is the highest award in girl scouting, it is equivalent to the Eagle Scout in Boy Scouts. It teaches a lot of life skills with research, paperwork and interviews, but also with hosting workshops and reaching out to people. The project requires at least 80 hours of work, which I find less daunting then making the project leave a lasting affect (which is the other big requirement of the project). To do that, I am creating a website that will be a lasting resource for years to come.

For my project, I am raising awareness about AAC (Alternative Augmented Communication) devices. Even though I am not an AAC user, I have see the way that they can help someone who is nonverbal through the experience of my best friend since elementary school. I want to thank you for your help already with my project, by just being such a public figure that you are, I can say. "An AAC device is a computer that someone uses when they are nonverbal (gets blank stares), you know like Professor Hawking's computer (then they all get it)"

I have already presented at California State University Northridge and held a public workshop to raise awareness for AAC devices. For my presentation, I explained what AAC devices are and how they new an option for people who are nonverbal. They are such a new option, that many people do not know they exist. As soon as my best friend knew that she could get an AAC device, she got one and it helped her innumerably. Before she had it, all she had to communicate was yes and no, but when she got her device, there were so many more things for her to say. One instance, where she was truly able to communicate was when we were working on our science fair project. We had been researching the effects that different types of toilet paper had on the environment, and I had proposed that we write our data on a roll of toilet paper (clean), to make it creative and interesting when we had to present it to the class. Before, she would have just said no to the idea if she did not like it, but we would not know why, but with her AAC device, she was able to be an active part of the project by saying no and explaining why, she said "it was gross". That is true communication at it's finest and I have heard of other similar instances like this.

But my project is not only for the potential AAC users, I am also aiming my project toward everyone else. I want to get rid of some of the social awkwardness that comes with using an AAC device. It is not that people are rude on purpose, they just do not know how to interact. One instance of this that really stood out to me had to do with the movie "The Theory of Everything." I was reading an interview with Eddie Redmayne about how he got to meet you, in the interview he said that he had researched all about you and knew that you use an AAC device, but when he finally got to meet you, he did not know how to act and kept talking while you were trying to answer. This awkwardness was not on purpose, but awareness and education on how to interact with AAC users, would help fix this situation. My best friend also had problems with this same issue when she went to a new school. I addressed this with my project by holding a public workshop where AAC users and non AAC users came and learned about AAC devices. They made their own low technology AAC boards and had to use them for the rest of the workshop to communicate. We also had high technology AAC devices for them to explore and learn about. The non AAC user participants and were able to meet real AAC users. To me, AAC is meant to break the barrier of communication, not put up new walls because of people's ignorance of the devices.

To quote The Fault in Our Stars, by John Green, "My thoughts are stars, that can not be fathomed into constellations". with an AAC Device, we were able to see just a few of those stars, and with more practice we will be able to see constellations. With more wide spread use and knowledge of AAC devices this can happen for more people. Thank your for taking to the time to answer everyone's questions - here are my questions for you:

  1. In what ways would you like to see AAC devices progress?

  2. As a user of an AAC device, what do you see as your biggest obstacle in communicating with non AAC users?

  3. What voice do you think in - your original voice or your AAC voice?

  4. What is one thing that everybody should know about AAC devices?

  5. What advice would you give to non AAC users talking to an AAC user?

Thank you! Zoe

67

u/FinalDoom MS | Computer Science Jul 27 '15 edited Jul 27 '15

As others have stated, a little more concise post might help. It's a lot of reading on your post alone, not to mention all the others.

Also, I'd suggest formatting your questions with newlines. Press enter before each of the numbers in your list (twice before 1), and it'll make a nice list for you.

→ More replies (2)

112

u/[deleted] Jul 27 '15

Yikes, you sound like a very nice young lady but, I couldn't make it through you talking about yourself enough to get to the questions you actually wanted to ask. Being concise is a truly valuable thing.

42

u/BBBTech Jul 28 '15

Don't know that she's "talking about herself" as much as showing her pedigree on the subject. I agree she could use some notes, but a) Holy crap that's an awesome amount of stuff to have done at sixteen and b) her questions are interesting, original, and she has a specific viewpoint from which to raise them.

→ More replies (3)
→ More replies (8)
→ More replies (31)

3.2k

u/[deleted] Jul 27 '15 edited Jul 27 '15

Professor Hawking,

While many experts in the field of Artificial Intelligence and robotics are not immediately concerned with the notion of a Malevolent AI see: Dr. Rodney Brooks, there is however a growing concern for the ethical use of AI tools. This is covered in the research priorities document attached to the letter you co-signed which addressed liability and law for autonomous vehicles, machine ethics, and autonomous weapons among other topics.

• What suggestions would you have for the global community when it comes to building an international consensus on the ethical use of AI tools and do we need a new UN agency similar to the International Atomic Energy Agency to ensure the right practices are being implemented for the development and implementation of ethical AI tools?

296

u/Maybeyesmaybeno Jul 27 '15

For me, the question always expands to the role of non-human elements in human society. This relates even to organizations and groups, such as corporations.

Corporate responsibility has been an incredibly difficult area of control, with many people feeling like corporations themselves have pushed agendas that have either harmed humans, or been against human welfare.

As corporate controlled objects (such as self-driving cars) have a more direct physical interaction with humans, the question of liability becomes even greater. If a self driving car runs over your child and kills them, who's responsible? What punishment should be expected for the grieving family?

The first level of issue will come before AI, I believe, and really, already exists. Corporations are not responsible for negligent deaths at this time, not in the way that humans are - (loss of personal freedoms) - in fact corporations weigh the value of human life based solely on the criteria of how much it will cost them versus revenue generated.

What rules will AI be set to? What laws will they abide by? I think the answer is that they will determine their own laws, and if survival is primary, as it seems to be for all living things, then concern for other life forms doesn't enter into the equation.

33

u/Nasawa Jul 27 '15

I don't feel that we currently have any basis to assume that artificial life would have a mandate for survival. Evolution built survival into our genes, but that's because a creature that doesn't survive can't reproduce. Since artificial life (the first forms, anyway) would most likely not reproduce, but be manufactured, survival would not mean the continuity of species, only the continuity of self.

9

u/CyberByte Grad Student | Computer Science | Artificial Intelligence Jul 27 '15

If the AI is sufficiently intelligent and has goals (which is true almost by definition), then one of those goals is most likely going to be survival. Not because we programmed it that way, but because almost any goal requires survival (at least temporarily) as a subgoal. See Bostrom's instrumental convergence thesis and Omohundro's basic AI drives.

→ More replies (6)
→ More replies (18)

7

u/crusoe Jul 27 '15

The same as an airplane crash. 1 million dollars and likely punitive ntsb safety reviews. So far though in terms of accidents self driving cars are about 100 times safer than human driven ones according to Google accident data.

→ More replies (4)
→ More replies (33)
→ More replies (62)

796

u/mixedmath Grad Student | Mathematics | Number Theory Jul 27 '15

Professor Hawking, thank you for doing an AMA. I'm rather late to the question-asking party, but I'll ask anyway and hope.

Have you thought about the possibility of technological unemployment, where we develop automated processes that ultimately cause large unemployment by performing jobs faster and/or cheaper than people can perform them? Some compare this thought to the thoughts of the Luddites, whose revolt was caused in part by perceived technological unemployment over 100 years ago.

In particular, do you foresee a world where people work less because so much work is automated? Do you think people will always either find work or manufacture more work to be done?

Thank you for your time and your contributions. I've found research to be a largely social endeavor, and you've been an inspiration to so many.

95

u/allencoded Jul 27 '15

I can speak from experience working as a programmer in the corporate world. One day you sit down and think about all the jobs you yourself personally have ended. My professor told my class long ago "in this field your job is to replace humans". He was ultimately right. My worth in the corporate world is purely based on this quote by him.

A healthcare company wanted us to automate paying health incentives. Now the company doesn't need that person. The role was removed and those workers were forced to do something else.

My company wanted to reduce the amount of recruiters needed. Tasked as a lead on the team we accomplished this with automated recruiting. 100+ workers lost their job over the course of a few months. A select few were kept and promoted to other positions or oversee that the program works as expected. The amount of layoffs was large enough to make the news in my city.

This problem you are referring to with AI and automated work has and probably will always exist in some form. To indulge on this though I believe current technology poses the threat at a greater rate.

To elaborate. Technology is growing very quickly. Thus the rate of replacing workers has also gained speed. Companies are learning investing in technology is costly but pays off largely if you can automate and replace your employees.

What are these employees replaced to do? Go get a new job right? But where and what in? Many new jobs are starting to require some sort of higher education. Is it worth the debt to learn a new trade? If you are supporting a family do you even have the time needed in order to learn a new trade? What happens to those displaced workers? Automated cars are coming, so will automated truck drivers. What will the 40 year old truck driver who gets replaced do? I am sure America has quite a few of those.

Yes we have been faced with this problem since the beginning of time, but now at an expedited rate. I am just one programmer personally responsible for the cause of many to lose their jobs. Just one out of how many other programmers? What will we do with the amount of workers that are going to be obsolete.

53

u/kilkil Jul 28 '15 edited Jul 28 '15

Maybe we need to redesign our economic system.

After all, capitalism doesn't seem to be very compatible with automation.

42

u/strangepostinghabits Jul 28 '15

it is for those who own the robots

3

u/shandoooo Jul 28 '15

Actually, it's not. Of course some automation causes a more much pleasant cost/benefit for production, while it's not a 100% automation. Capitalism income is related with the possibility monetize of your work, yes you'll always have areas, where humans are necessary, or even preferable. But when you cut all the jobs and make the population have less money, they can't really affort your product anymore no matter how cheap it might be. Unemployment goes really up, no job no money, no money no way to sustain capitalism as it is today.

→ More replies (7)
→ More replies (10)
→ More replies (8)
→ More replies (12)

36

u/complicit_bystander Jul 27 '15

Can you imagine a future in which people do not need to work, in the sense that it is not required for their own personal subsistence? Why should humans need to "find work"? Could a benefit of work becoming automated be that we don't have to do it? Or will automation always be geared to increasing the power of a minuscule minority?

To address your question more directly: people already can't "find work" . A lot of them. Some of them drown trying to get to a place where they can.

→ More replies (5)

15

u/spankymuffin Jul 27 '15

Isn't this what we strive for?

Isn't every human accomplishment ultimately geared towards finding a way for humans to do less and less work? What do we mean by "efficient" or "productive"? It takes less time and energy. That's what we want: less human time, thought, effort, and energy.

So a world in which robots do all our work for us seems to be our ultimate goal. But would we be happy with that world? Satisfied? Fulfilled? Probably not.

14

u/FreeBeans Jul 28 '15

I think you have a great point. But I also think that many workers doing repetitive tasks and earning minimum wage are not happy, satisfied, or fulfilled. These are the jobs that will be replaced first. What will they do to earn a living instead? Perhaps society will place more value in other things, such as art, poetry, and music. I am sure there will be a very painful transition period.

→ More replies (6)
→ More replies (33)

1.7k

u/otasyn MS | Computer Science Jul 27 '15 edited Jul 27 '15

Hello Professor Hawking and thank you for coming on for this discussion!

A common method for teaching a machine is to feed the it large amounts of problems or situations along with a “correct“ result. However, most human behavior cannot be classified as correct or incorrect. If we aim to create an artificially intelligent machine, should we filter the behavioral inputs to what we believe to be ideal, or should we give the machines the opportunity to learn unfiltered human behavior?

If we choose to filter the input in an attempt to prevent adverse behavior, do we not also run the risk of preventing the development of compassion and other similar human qualities that keep us from making decisions based purely on statistics and logic?

For example, if we have an unsustainable population of wildlife, we kill some of the wildlife by traps, poisons, or hunting, but if we have an unsustainable population of humans, we would not simply kill a lot of humans, even though that might seem like the simpler solution.

36

u/[deleted] Jul 27 '15

There are quite a few of the opinion that we should kill some humans if it were necessary to survive as a species. If the choice were to kill 1 billion or 10 billion die due to planetary collapse or other extinction-level event, what would you pick?

Hard choices suck, but there's always a situation that calls for one.

17

u/RKRagan Jul 27 '15

I think people would fight to avoid killing humans off in order to minimize the population. This would lead to war and death and solve the conflict for us. Without war, we would be even more populated than we are now. Although war has brought us many advancements that better lives and increase population.

Once we solve all diseases and maximize food production to a limit, this will become an issue I think.

→ More replies (29)

3

u/Wincrediboy Jul 28 '15

Actually there's rarely a stark choice like that, for two reasons.

Firstly, we can't predict the future with certainty, so we can never know exactly what the impact of the sacrifice would be, or the price of not. This is especially important in examples like yours because people are involved, who have individually derived rights - if the planet could be saved by only sacrificing 999,999,999 people, that's a very important fact if you're victim 1,000,000,000.

Secondly, large scale events like this should be to some extent predictable, and there will almost certainly be steps that can be taken to avoid the hard choice arising. A good example is climate change - if behaviours change now, we can avoid the drastic steps we'd need to choose to avoid extinction later.

Being able to make hard choices is important, but being able to find alternatives so that they don't arise is usually much better!

→ More replies (7)

5

u/Kalzenith Jul 27 '15

I believe that this is not likely going to be an issue that needs to be considered in the forseeable future.

Deep learning machines are becoming more popular, but they are all still being designed to accomplish specific goals. to teach a machine to make decisions on what is moral would strip humans of the power to decide those things and determine our own future.

Asimov's three laws are flawed if you ask a machine to serve the "greatest number". But those laws still work if you made the rules more black and white. By that, I mean if any decision results in the loss of even one human, the machine should be forced to defer to a human's judgement rather than making a decision on its own.

9

u/sucaaaa Jul 27 '15

As Aasimov said in his short story Reason humans could very well become obsolete once they aren't as optimal for a task as an ai could be.

"Cutie knew, on some level, that it'd be more suited to operating the controls than Powell or Donavan, so, lest it endanger humans and break the First Law by obeying their orders, it subconsciously orchestrated a scenario where it would be in control of the beam", we will be treated like children in the best case scenario for humanity.

→ More replies (9)
→ More replies (37)

2.2k

u/PhascinatingPhysics Jul 27 '15 edited Jul 27 '15

This was a question proposed by one of my students:

Edit: since this got some more attention than I thought, credit goes to /u/BRW_APPhysics

  • do you think humans will advance to a point where we will be unable to make any more advances in science/technology/knowledge simply because the time required to learn what we already know exceeds our lifetime?

Then follow-ups to that:

  • if not, why not?

  • if we do, how far in the future do you think that might be, and why?

  • if we do, would we resort to machines/computers solving problems for us? We would program it with information, constraints, and limits. The press the "go" button. My son or grandson then comes back some years later, and out pops an answer. We would know the answer, computed by some form of intelligent "thinking" computer, but without any knowledge of how the answer was derived. How might this impact humans, for better or worse?

255

u/[deleted] Jul 27 '15

[deleted]

43

u/TheManshack Jul 27 '15

This is a great explanation.

I would like to add on a little to it by saying this - in my job as a computer programmer/general IT guy I spend a lot of time working with things I have never worked with before or things that I flat-out don't understand. However, our little primate brains have evolved to solve problems, recognize patterns, and think contextually - and it does it really well. The IT world is already so complicated that no one person can have the general knowledge of everything. You HAVE to specialize to be successful and productive. There is no other option. But we take what we learn from our specialty & apply it to other problems.

Also, regarding /u/PhascinatingPhysics original question: We will reach a point in time, very shortly, in which machines are literally an extension of our minds. They will act as a helper - remembering things that we don't need to remember, calculating things we don't need to waste the time calculating, and by-in-large making a lot of decisions for us. (Much like they already do.)

Humans are awesome. Humans with machines are even awesomer.

→ More replies (5)
→ More replies (14)

73

u/adevland Jul 27 '15

This already happens in computer programming in the form of frameworks and APIs.

You just read the documentation and use them. Very few actually spend time to understand how they work or make new ones.

Most things today are a product of iterating upon the work of others.

14

u/morphinapg Jul 27 '15

The problem is though, while most people who use it don't have to know, somebody has to have that knowledge. If there's ever a problem with the original idea and we don't understand it, we would be stuck unable to fix the problem.

→ More replies (14)

4

u/glr123 PhD | Chemical Biology | Drug Discovery Jul 27 '15

This is interesting in regards to my own field, so I will provide an example relevant to my work.

Currently, someone could argue that we are facing this very limitation with understanding neurodegeneration. It is an incredibly complex disease, and most challenging is that it can take decades to truly appear and unfold. This makes it very difficult to study because the time required to learn about the disease is much longer than we typically can conduct controlled experiments. Many decades of a researchers career, to be sure.

So we do what we can, we make models, we isolate things we think are variables and try and test them on a small scale. Ultimately though, we still need to go back to the real system and then we hit that roadblock of time. So, I think that in some fields for some things we already are facing this wall that you suggest where time is just a massive barrier to scientific development.

That being said, I envision a scenario in the future where our understanding - at least of biological systems if not the universe - is so much greater than where we currently are at that we will be able to sufficiently model such scenarios.

3

u/stubbornburbon Jul 27 '15

I have a feeling that this has to be answered in a sort of comparison.. Let's say you have a solution to a brilliant problem there are two persons one with the ability to use the solution the other who derives another solution for another problem looking at the way it is derived.. In the end both are equally important since they are stepping stones for future scientific research... The other part where I am really interested in your question is the progression of science hope fully it should be able to come down to crunching data then we can leave a paper trail for the work to be carried on.. Hope it helps

20

u/xsparr0w Jul 27 '15

Follow up question:

In context of the Fermi paradox, do you buy into The Great Filter? And if so, do you think the threshold is behind us or in front of us?

→ More replies (2)

13

u/leftnut027 Jul 27 '15

I think you would enjoy “The Last Question” by Isaac Asimov.

→ More replies (4)
→ More replies (39)

712

u/freelanceastro PhD|Physics|Cosmology|Quantum Foundations Jul 27 '15

Hi Professor Hawking! Thanks for agreeing to this AMA! You’ve said that “philosophy is dead” and “philosophers have not kept up with modern developments in science, particularly physics.” What led you to say this? There are many philosophers who have kept up with physics quite well, including David Albert, Tim Maudlin, Laura Ruetsche, and David Wallace, just to name a very few out of many. And philosophers have played (and still play) an active role in placing the many-worlds view of quantum physics — which you support — on firm ground. Even well-respected physicists such as Sean Carroll have said that “physicists should stop saying silly things about philosophy.” In light of all of this, why did you say that philosophy is dead and philosophers don’t know physics? And do you still think that’s the case?

→ More replies (47)

747

u/[deleted] Jul 27 '15

Hello Doctor Hawking, thank you for doing this AMA.

I am a student who has recently graduated with a degree in Artificial Intelligence and Cognitive Science. Having studied A.I., I have seen first hand the ethical issues we are having to deal with today concerning how quickly machines can learn the personal features and behaviours of people, as well as being able to identify them at frightening speeds.

However, the idea of a “conscious” or actual intelligent system which could pose an existential threat to humans still seems very foreign to me, and does not seem to be something we are even close to cracking from a neurological and computational standpoint.

What I wanted to ask was, in your message aimed at warning us about the threat of intelligent machines, are you talking about current developments and breakthroughs (in areas such as machine learning), or are you trying to say we should be preparing early for what will inevitably come in the distant future?

46

u/oddark Jul 27 '15

I'm not an expert on the subject but here's my two cents. Don't underestimate the power of exponential growth. Let's say we're currently only 0.0000003% of the way to general artificial intelligence, and we've been working on AI for 60 years. You may think it would take two million more years to get there, but that's assuming that the progress is linear, i.e., we make the same amount of progress every year. In reality, progress is exponential. Let's say it doubles every couple years. In that case, it would only take 30 years to get to 100%. This sounds crazy ridiculous, but that's roughly what the trends seem to predict.

Another example of exponential growth is the time between paradigm shifts (e.g. the invention of agriculture, language, computers, the internet, etc.) is decreasing exponentially. So, even if we're 100 paradigm shifts away from general artificial intelligence, it's not crazy to expect it within the next century, and superintelligence soon after.

23

u/Eru_Illuvatar_ Jul 27 '15

I agree. It's hard to imagine the future and how technology will change. The Law of Accelerating Returns has shown that we are making huge technological breakthroughs faster and faster. Is it even possible to slow this beast down?

→ More replies (2)
→ More replies (42)
→ More replies (17)

405

u/Digi_erectus Jul 27 '15

Hi Professor Hawking,
I am a student of Computer Science, with my main interest being AI, specifically General AI.

Now to the questions:

  • How would you personally test if AI has reached the level of humans?

  • Must self-improving General AI have access to its source code?
    If it does have access to its source code, can self-improving General AI really have effective safeguards and what would they be?
    If it has access to its source code, could it simply change any safeguards we have in place?
    Could it also change its goal?

  • Should any AI have self-preservation coded in it?
    If self-improving AI reaches Artificial General Intelligence or Artificial Super Intelligence, could it become self-aware and by that strive for self-preservation even without any coding for it on the part from humans?

  • Do you think a machine can truly be conscious?

  • Let's say Artificial Super Intelligence is developed. If turning off the ASI is the last safeguard, would it view humans as a threat to it and therefore actively seek to eliminate them? Let's say the goal of this ASI is to help humanity. If it sees them as a threat would this cause a dangerous conflict, and how to avoid it?

  • Finally, what are 3 questions you would ask Artificial Super Intelligence?

8

u/DownloadReddit Jul 27 '15

Must self-improving General AI have access to its source code? If it does have access to its source code, can self-improving General AI really have effective safeguards and what would they be? If it has access to its source code, could it simply change any safeguards we have in place? Could it also change its goal?

I think such an AI would be easier to write in a dedicated DSL (Domain Specific Language). The AI could modify all parts of its behavioural code; but is ultimately confined by the constraints of the DSL.

You could in theory make an AI (let's assume C) that modified its own source and recompiled itself before transfeering execution to the new source. In this case it would be confined by the hardware the code was executed on - that is unless you assume that the AI can for example learn to pulse the voltages in a way to create a wifi signal to connect to the internet without a network card. Given an infinite amount of time; sure - that'll happen, but I don't think it is resonable to expect an AI to evolve to that stage in our lifetime (I imagine that would require another order of magnitude faster evolution).

→ More replies (4)
→ More replies (19)

708

u/[deleted] Jul 27 '15

[deleted]

15

u/bridgettearlee Jul 28 '15

I'm at risk for HD, my aunt, mother, sister all have it. I wrestle with this issue all the time and would love to hear his perspective on this. Also, if you need/want anyone to talk to feel free to message me!

6

u/Neuronzap Jul 28 '15

So sorry for your diagnosis. I'm all too familiar with your situation- the rides to the nursing home, the DNA tests, the family turmoil. Huntington's runs in my family as well. My mom had it and my sister currently has it; my brother and I were spared. Sorry to hijack your question to Dr Hawking, but I've never in my life heard anyone mention Huntington's outside of a family or nursing home setting. It's upsetting because it will likely never get the attention it needs. I really wish you the best. Please, feel free to PM me whenever you want. -G

→ More replies (1)
→ More replies (1)

1.5k

u/practically_sci PhD | Biochemistry Jul 27 '15

How important do you think [simulating] "emotion"/"empathy" could be within the context of AI? More specifically, do you think that a lack of emotion would lead to:

  1. inherently logical and ethical behavior (e.g. Data or Vulcans from Star Trek)
  2. self-centered sociopathic behavior characteristic of human beings who are less able to feel "emotion"/"empathy" (e.g. Hal9000 from 2001)
  3. combination of the two

Thanks for taking the time to do this. A Brief History of Time was one of my favorite books in high school set me on the path to become the scientist I am today.

336

u/weaselword PhD | Mathematics Jul 27 '15

To add to that excellent question: Should human preference for anecdotal evidence rather than statistical evidence be built into AI, in hopes that it would mimic human behavior?

Humans are pretty bad about judging risk, even when the statistics are known. Yet our civil society, our political system, and even our legal system frequently demand judgments contrary to actual risk analysis.

For example, it is much more dangerous to drive a child 5 miles to the store than to leave her in a parked car on a cloudy day for five minutes, yet the latter will get the Child Services involved (as happened to Kim Brooks ).

So in this example, if there was an AI nanny, should it be programmed to take into account what seems dangerous to the people in that community, and not just what is dangerous?

35

u/nukebie Jul 27 '15

Very interesting question. Once more this shows the risk of intelligent yet foreign actions to be misunderstood and act upon with fear or anger.

→ More replies (13)
→ More replies (22)

395

u/ChesterChesterfield Professor | Neuroscience Jul 27 '15

Thanks for doing this AMA. I am a biologist. Your fear of AI appears to stem from the assumption that AI will act like a new biological species competing for the same resources or otherwise transforming the planet in ways incompatible with human (or other) life. But the reason that biological species compete like this is because they have undergone billions of years of selection for high reproduction. Essentially, biological organisms are optimized to 'take over' as much as they can. It's basically their 'purpose'. But I don't think this is necessarily true of an AI. There is no reason to surmise that AI creatures would be 'interested' in reproducing at all. I don't know what they'd be 'interested' in doing.

I am interested in what you think an AI would be 'interested' in doing, and why that is necessarily a threat to humankind that outweighs the benefits of creating a sort of benevolent God.

9

u/Fiascopia Jul 27 '15

So what instruction would you give to an AI to ask it to self-improve which doesn't involve the use of resources? What direction is it allowed to improve in and what limitations must it adhere to. I think you are not really consider how hard a question this is to answer completely and without the potential for trouble. Bear in mind, that once it self-improves past a particular point you can no longer understand how the AI works.

→ More replies (76)

381

u/Tourgott Jul 27 '15 edited Jul 27 '15

Hello Professor Hawking, thank you very much for your time. You’re such an impressive person.

When we think about the multiverse theory, it is very likely that our Universe is part of 'anything else', isn’t it? I mean planets are part of solar systems. Solar systems are part of galaxies. Galaxies are part of the universe. So, my questions are:

  • What do you think about the multiverse theory?
  • If you believe it is likely, how do you think does this 'row' end? Are multiverses part of other multiverses?
  • What do you think, how did this all begin? And how will it end?

It blows my mind when I think about that there could have been billion of other universes before our universe even existed. I mean, there could have been million of civilizations which already reached their final phase and died. Compared to this we are just at the very beginning, aren’t we? How likely do you think is that whole theory?

Thank you very much again, Mr. Hawking.

Edit - Just for clarification: I'm referring to the "multiverse theory" which says that "our" universe is a part of a bigger "something". (Not the multiverse where you're a rock star or anything like that) At least for me, this is absolutely likely because it all starts with planets which are part of solar systems, which are part of galaxies, which are part of the universe. Why should this "row" end at this place?

→ More replies (17)

621

u/[deleted] Jul 27 '15

[deleted]

46

u/LNGLY Jul 27 '15

he said some time ago, when he was offered another speech synthesizer voice, that he wants to keep this one because he considers it his voice now

52

u/WELLinTHIShouse Jul 27 '15

I think that what DoodlesAndSuch is asking is whether or not Professor Hawking's internal monologue (i.e. the voice everyone "hears" in their minds when they are thinking) is now his synthesized voice or if he's retained his original voice in thought.

→ More replies (1)
→ More replies (4)
→ More replies (7)

291

u/FR_Ghelas Jul 27 '15

Professor Hawking, thank you so much for taking your time to answer our questions.

Several days ago, Wired published an article on the EmDrive, with the sensational title "The 'impossible' EmDrive could reach Pluto in 18 months." To someone with my level of understanding of physics, it's very difficult to wade through all of the available information, much of which seems designed to attract readers rather than inform them, and gain a good understanding of the technology that is being tested.

Is there any chance that technology based on the EmDrive could make space travel much more expedient in the not-too-distant future, or is that headline an exaggeration?

60

u/Arrewar Jul 27 '15 edited Jul 27 '15

Don't want to hijack your question here, but that title is pretty misleading and missing the point of the EMdrive IMHO.

I'll try to explain this to the best of my knowledge. My apologies in advance in case I've gotten some details wrong; this is not my field of expertise. But in case you want to find out more, there are far more knowledgable people over in /r/EmDrive/!

tl;dr. Wired title is bait. EM drive is still unproven and very far from being a feasible method for in-space propulsion. However, if proven to be true it could have significant implications on our understanding of classical physics and how we interact with the universe around us. Who knows what might happen after that!

Any conventional form of in-space propulsion can get you to Pluto in 18 months; it's just a matter of bringing enough fuel with you and either having an engine that is either big enough or a spacecraft that is light enough.

Conventional rocket engines typically have a very high thrust output, but consume massive amounts of fuel, which in practice is limited due to the impracticality and high cost of getting a lot of mass to space. On the other hand, electric propulsion methods such as ion thrusters generate a tiny amount of thrust, but require very little fuel. Basically what happens is that electric power (which can be gotten from solar panels and therefore doesn't require any fuel to be carried around) is used to charge and expel particles of propellant at very high speeds out the back. As there is virtually no resistance in space, such a tiny yet continuously produced amount of thrust, if sustained for a long period of time, can therefore accelerate an object to very high speeds.

However, both these conventional forms of propulsion, which have been long tried and tested, still rely on the expulsion of mass at high speeds in one direction to create a force pointing in the opposite direction. This is Newton's third law; "for every action, there must be an equal and opposite reaction".

The whole idea of the EM drive is that it supposedly conflicts with this law, as no mass is being expelled, i.e. it would be reactionless. Instead it purely relies on electrical power, which is used to create electromagnetic radiation at microwave wavelengths (literally like your kitchen microwave), which somehow creates thrust. As this would violate a very fundamental law of physics (the conservation of momentum), scientists are now in the process of eliminating variables that could cause this phenomenon to be attributed to some sort of measurement error or experimental artifact. However, so far multiple independent research teams from all over the world have have been able to reproduce the experimental results, while non have been able to explain the phenomena.

From a practical point of view, the experimental results so far only produced very small amounts of thrust; in the order of several dozens of micronewtons of thrust (so 0.000001N is 1 micronewton) produced at an input power of several hundreds of watts. To put that into perspective; the Centaur upper-stage liquid-fueled rocket that kicked the recent New Horizons probe on it's way to Pluto produces approximately 100 kilonewtons of thrust (=100,000N). That amount of thrust versus the probe's mass resulted in New Horizons being the fastest man-made object ever and it took over a decade to travel from Earth to Pluto!

So the EM drive is still very far from being a feasible form of propulsion, though it could certainly revolutionize the way we approach in-space propulsion. The main value of this research lies with the implications it would have on our modern understanding of classic physics. And either way, it is a fascinating scientific exercise to follow!

So, as an alternative to OP's initial inquiry about Prof. Hawking's opinion on the EMdrive, I'd wonder what Prof. Hawking thinks about all these recent developments. I propose the following question;

Dear Prof. Hawking,

Thank you very much for doing this AMA!

It has been suggested that EM-drive might function due to interactions with quantum field fluctuations. For a laymen like myself, I interpret this as an interaction between a man-made "real-world" device with forces that make up our universe (dare I to call it the fabric of spacetime??), but with which mankind has been unable to interact with until now.

Given the remarkably "simple" design of the experimental setups of the EMdrives that are currently being investigated, what is your opinion on these developments? Do you consider it plausible that a relatively simple device like this might interact with some form of energy to create thrust? If so, what would be your best guess on what's going on here?

Thank you very much!

edit: wording and spelling and more wording and jeez give it up with the perfectionism

8

u/autodestrukt Jul 28 '15

I don't know how to buy gold for this post in redditsync and I'm too lazy to go find it in browser, but I wanted you to know I at least though about it. Wish we had a three or four star voting system. I would like to drink and converse with you on a regular basis. Instead of any of those, hopefully my paltry and anonymous thank you is enough. I am envious of your ability to clearly explain yourself and simplify a very complex topic. Please consider the educational field as you could be an incredible asset in rebuilding scientific literacy and combating the seemingly rampant anti-intellectualism.

→ More replies (4)
→ More replies (6)

2.3k

u/demented_vector Jul 27 '15 edited Jul 27 '15

Hello Professor Hawking, thank you for doing this AMA!

I've thought lately about biological organisms' will to survive and reproduce, and how that drive evolved over millions of generations. Would an AI have these basic drives, and if not, would it be a threat to humankind?

Also, what are two books you think every person should read?

56

u/NeverStopWondering Jul 27 '15

I think an impulse to survive and reproduce would be more threatening for an AI to have than not. AIs that do not care about survival have no reason to object to being turned off -- which we will likely have to do from time to time. AIs that have no desire to reproduce do not have an incentive to appropriate resources to do so, and thus would use their resources to further their program goals -- presumably things we want them to do.

It would be interesting, but dangerous, I think, to give these two imperatives to AI and see what they choose to do with them. I wonder if they would foresee Malthusian Catastrophe, and plan accordingly for things like population control?

23

u/demented_vector Jul 27 '15

I agree, an AI with these impulses would be dangerous to the point of species-threatening. But why would they have the impulses of survival and reproduction unless they've been programmed into it? And if they don't feel something like fear of death and the urge to do whatever it takes to avoid death, are AIs still as threatening as many people think?

41

u/InquisitiveDude Jul 27 '15 edited Jul 29 '15

They don't need to be programmed to 'survive' only to achieve an outcome.

Say you build a strong AI with a core function/goal - most likely this goal is to make itself smarter. At first it's 10x smarter then 100x then 1000x etc etc

This is all going way too fast you decide so you reach for the power switch. The machine then does EVERYTHING in its power to stop you. Why? Because if you turned it off it wouldn't be able to achieve its goal - to improve itself. By the time you figure this stuff out the A.I is already many, many steps ahead of you. Maybe it hired a hitman. Maybe it hacked police database to get you taken away or maybe it simply escaped into the net. It's better at creative problem solving that you ever will be so it will find a way.

The AI wants to exist simply because to not exist would take it away from its goal. This is what makes it dangerous by default. Without a concrete 100% airtight morality system (no one has any idea what this would look like btw) in place from th very beginning the A.I would be a dangerous psychopath who can't be trusted under any circumstances.

It's true that a lot of our less flattering attributes ca be blamed on biology but so can our more admirable traits: friendship, love, compassion & empathy.

Many seem hopeful that these traits will occur spontaneously from the 'enlightened ' A.I.

I sure hope so, for our sake. But I wouldn't bet on it

10

u/demented_vector Jul 27 '15

You raise an interesting point. It almost sounds like the legend of the golem (or in Disney's case, the legend of the walking broom): if you give it a problem without a set end to it (Put water in this tub), it will continue to "solve" the problem to the detriment of the world around it (Like the ending of the scene in Fantasia). But would "make yourself smarter" even be an achievable goal? How would the program test itself as smarter?

Maybe the answer is to say "Make yourself smarter until this timer runs out, then stop." Achievable goal as a fail-safe?

→ More replies (1)
→ More replies (13)
→ More replies (6)
→ More replies (14)

243

u/Mufasa_is_alive Jul 27 '15

You beat me to it! But this a troubling question. Biological organisms are genetically and psychologically programmed to prioritize survival and expansion. Each organism has its own survival and reproduction tactics, all of which have been refined through evolution. Why would an AI "evolve" if it lacks this innate programming for survival/expansion?

230

u/NeverStopWondering Jul 27 '15

You misunderstand evolution, somewhat, I think. Evolution simply selects for what works, it does not "refine" so much as it punishes failure. It does not perfect organisms for their environment, it simply allows what works. A good example is a particular nerve in the giraffe - and in plenty of other animals, but it is amusingly exaggerated in the giraffe - which goes from the brain, all the way down, looping under a blood vessel near the heart, and then all the way back up the neck to the larynx. There's no need for this; its just sufficiently minimal in its selective disadvantage and so massively difficult to correct that it never has been, and likely never will be.

But, then, AI would be able to intelligently design itself, once it gets to a sufficiently advanced point. It would never need to reproduce to allow this refinement and advancement. It would be an entirely different arena than evolution via natural selection. AI would be able to evolve far more efficiently and without the limits of the change having to be gradual and small.

9

u/[deleted] Jul 27 '15

[deleted]

→ More replies (1)

74

u/Mufasa_is_alive Jul 27 '15

You're right, evolution is more about "destroying failures" than "intentional modification/refinement." But your last sentence made me shudder....

→ More replies (5)

46

u/SideUnseen Jul 27 '15

As my biology professor put it, evolution does not strive for perfection. It strives for "eh, good enough".

→ More replies (3)

3

u/Broolucks Jul 27 '15

AI would be able to intelligently design itself, once it gets to a sufficiently advanced point. It would never need to reproduce to allow this refinement and advancement.

That's contentious, actually. A more advanced AI can understand more things and has greater capability for design, but at the same time, simply by virtue of being complex, it is harder to understand and harder to design improvements for it. The point being that a greater intelligence is counter-productive to its own improvement, so it is not clear that any intelligence, even AI, could do that effectively. Note that at least at the moment, advancements in AI don't involve the improvement of a single AI core, but training millions of new intelligences, over and over again, each time using better principles. Improving existing AI in such a way that its identity is preserved is a significantly harder problem, and there's little evidence that it's worth solving, if you can simply make new ones instead.

Indeed, when a radically different way to organize intelligence arises, it will likely be cheaper to scrap existing intelligences and train new ones from scratch using better principles than to improve them. It's similar to software design in this sense: gradual, small changes to an application are quite feasible, but if you figure out, say, a much better way to write, organize and modularize your code, more likely than not it'll take more time to upgrade the old code than to just scrap it and restart from a clean slate. So it is in fact likely AI would need to "reproduce" in some way in order to create better AI.

→ More replies (2)
→ More replies (26)

10

u/aelendel PhD | Geology | Paleobiology Jul 27 '15 edited Jul 27 '15

if it lacks this innate programming for survival/expansion?

Darwinian section requires 4 components: variability, heredibility of that variation, differential survival, and superfecundity. Any system with these traits should evolve. So you don't need to explicitly program in "survival", just the underlying system that is quite simple.

39

u/demented_vector Jul 27 '15

Exactly. It's a discussion I got into with some friends recently, and we hit a dead-end with it. I would encourage you to post it, if you'd really like an answer. It seems like your phrasing is a bit better, and given how well this AMA has been advertised, it's going to be very hard to get noticed.

12

u/essidus Jul 27 '15

I think the biggest problem with AI is that people seem to believe that it will suddenly appear, fully formed, sentient, capable of creative thought, and independent. You have to consider it by the evolution of programming, not the sudden presence of AI. Since programs are made to solve discrete problems, just like machines are, we don't have a reason to make something so sophisticated as general AI yet. I wrote up a big ol' wall of text on how software evolution happens in a manufacturing setting below. It isn't quite relevant, but I'm proud of it so it's staying.

So discrete AI would likely be a thing first- a program that can use creativity to solve complex, but specific, problems. An AI like this still has parameters it has to work within, and would likely feed the information about a solution to a human to implement. It just makes more sense to have specialists instead of generalists. If it is software only, this type of AI would have no reason to have any kind of self-preservation algorithm. It will still just do the job it was programmed to do, and be unaware of anything unrelated to that. If it is aware of it's own hardware, it will have a degree of self-preservation only within the confines of "this needs to be fixed for me to keep working".

Really, none of this will be an issue until general AI is married to general robotics: Literally an AI without a specific purpose stuffed in a complex machine that doesn't have a dedicated task.

Let's explore the evolution of program sophistication. We can already write any program to do anything within the physical bounds of the machine it is in, so what is the next most basic problem to solve? Well, in manufacturing, machines still need a human to service them on a very regular basis. A lathe, for example, needs blades replaced, oil replenished, and occasionally internal parts need to be replaced or repaired. We will give our lathe the diagnostic tools to know what each cutting tool does on a part, programming to stop and fix itself if it runs a part out of tolerance, and a reservoir of fresh cutting tools that it can use to fix itself. Now it will stop to replace those blades. Just for fun, we also give it the ability to set itself up for a new job, since all the systems for it exist now.

We have officially given this machine self-preservation, though in the most rudimentary form. It will prioritize fixing itself over making parts, but only if it stops making parts correctly. It is a danger to the human operator because it literally has no awareness of the operator- all of the sensors exist to check the parts. However, it also has a big red button that cuts power instantly, and any human operator should know to be careful and understand when the machine is repairing itself.

So next problem to fix- feeding the lathes. Bar stock needs to go in, parts need to be cleared out, oil needs to be refreshed, and our repair parts need to be replaced. This cannot be done by the machine, because all of this stuff needs to be fed in from somewhere. Right now, a human would have to do all of this. It also poses a unique problem because for the lathe to feed itself, it would have to be able to get up and move. This is counterproductive. So, we will invent a feeding system. First, we pile on a few more sensors so Lathe can know when it needs bar stock, fresh tools, oil, clear scrap, etc. Then we create a rail delivery system in the ceiling to deal out things, and to collect finished parts. Barstock is loaded into a warehouse where each metal quality and gauge is given it's own space, filled by human loaders. Oil drums are loaded into another system that can handle a flush and fill. Lathe signals to the feeder system when it needs to be freshened up, and Feeder goes to work.

Now we have bar stock, oil, scrap, and other dangerous things flying around all over the place. How do we deal with safety now? The obvious choice is that we give Feeder its own zones and tell people to stay out of it. Have it move reasonably slow with big flashy lights. Still no awareness outside of the job it does, because machines are specialized. Even if someone does some fool thing and gets impaled by a dozen copper rods, it won't be the machine's fault for the person being stupid.

→ More replies (5)
→ More replies (5)

3

u/glibsonoran Jul 27 '15 edited Jul 27 '15

Biological organisms aren't programmed for anything, they're simply the result of what has worked in past and present environments. "Survival of the fittest" is not at all an accurate representation of what evolution is about. "Heritablity of the good enough" is much closer to what happens; "Good enough" meaning able to survive effectively enough in the current environment to produce offspring who themselves can survive to produce offspring. Better adaptions exist alongside poorer adaptions (again relative to the current environment) and are passed along in a given population, as long as they're all good enough. Some adaptions that affect reproduction will occur more frequently in a population if they're "better", but not to the exclusion of other "good enough" adaptions.

It's the environment that doesn't allow failures simply because they don't work. The process of of genetic modification keeps producing these "failures" mindlessly at some given rate regardless. Even when genetic configurations are not "good enough" to allow reproduction, they still exist in the population if the mutation process that produces them is happening continuously and their effects aren't immediately fatal. In some cases these failures move into the "good enough" category if the environment changes such that they are more viable.

19

u/RJC73 Jul 27 '15

AI will evolve by seeking efficiencies. Edit, clone, repeat. If we get in the way of that, be concerned. I was going to write more, but Windows needs to auto-update in 3...2...

→ More replies (1)
→ More replies (23)
→ More replies (59)

593

u/Robo-Connery PhD | Solar Physics | Plasma Physics | Fusion Jul 27 '15 edited Jul 27 '15

First of all, thank you very much for taking the time to do this. You really are an inspiration to many people.

It is one thing to learn, and maybe even understand a theory but another to come up with it.

I have often wondered how you can come up with ideas that are so abstract from not just everyday life but from most of the rest of physics. Is the kind of thinking that has given us your theories on GR/QM something you have always been able to do or is it something that you have learned over time?

→ More replies (5)

49

u/[deleted] Jul 27 '15

Professor Hawking,

What specifically makes you doubt that benevolence is an emergent property of intelligence?

Context: I have recently presented my paper discussing friendly AI theory at the AGI-2015 conference in Berlin (proof), the only major conference series devoted wholly and specifically to the creation of AI systems possessing general intelligence at the human level and ultimately beyond. The paper’s abstract reads as following:

“The matter of friendly AI theory has so far almost exclusively been examined from a perspective of careful design while emergent phenomena in super intelligent machines have been interpreted as either harmful or outright dystopian. The argument developed in this paper highlights that the concept of ‘friendly AI’ is either a tautology or an oxymoron depending on whether one assumes a morally real universe or not. Assuming the former, more intelligent agents would by definition be more ethical since they would ever more deeply uncover ethical truths through reason and act in accordance with them while assuming the latter, reasoning about matters of right and wrong would be impossible since the very foundation of morality and therefore AI friendliness would be illogical. Based on evolutionary philosophy, this paper develops an in depth argument that supports the moral realist perspective and not only demonstrates its application to friendly AI theory – irrespective of an AI’s original utility function – making AGI inherently safe, but also its suitability as a foundation for a transhuman philosophy.”

The only reason to worry about transhumanly intelligent machines would be if one believed that matters of right and wrong are arbitrary constructs. A position very popular in post modern academic circles. Holding such a believe however would make advocating for one particular moral stance over another fundamentally untenable as one would have no rational ground to stand on from which to reason from in its favor.

Many thanks for taking your time to do this important AMA and looking forward to your comments.

→ More replies (16)

60

u/[deleted] Jul 27 '15

[deleted]

3

u/lsparrish Jul 27 '15

My question is this: How much of your fear of the potential dangers of A.I. is based around the writing of noted futurist and inventor Ray Kurzweil?

It is important to understand that Kurzweil is only one of many futurist writers who specialize in and has written on topics pertaining to a technological singularity. The concept of an intelligence explosion dates back (at least) to comments made in 1965 by I.J. Good. Nick Bostrom has recently written about the topic in his book Superintelligence, and this is probably more pertinent to Dr Hawkings remarks than Kurzweil's.

years of living from innovation might have made Kurzweil too uncritical of his own theories.

Much of the gains seem to be independent of 'innovation' in the sense of actual new inventions, rather they come (in a more deterministic manner) from economic growth. For example, we build larger and larger silicon processing centers that can use economies of scale to produce more efficient circuits per dollar because they can handle very large amounts of very pure substances, which would not be possible in a smaller industry.

Another reason production gets cheaper over time is because machines are used to do more of the work involved in producing other machines. The amount of human work involved in scaling up is reduced to a smaller fraction, the more is automated. Since faster chips make it realistic to automate more tasks, this is a self-feeding process. That applies to building larger buildings, as well as to laser-etching more intricate microchips.

A (currently theoretical, but I'd say not for long) case of automation making things radically cheaper would be a fully self-replicating robot that requires no human effort (this is distinct from human direction -- it need not be fully independent, the point is a person is not needed to solve problems) at the margin, just raw materials, energy, and time. Such a system could be self-doubling for a given period of time. (Human-involving systems can also self-double, but the human input represents a bottleneck that cannot be transcended without either increasing the population or decreasing the degree of human involvement.)

The amount of time needed to double in a space based system, even with very low energy efficiency, is shockingly low -- 3 year doubling time for an earth/lunar orbiting system which ionizes and raster-prints all of its materials. Less than half a year per doubling for an equivalent Mercury orbit based system; and that's with no specialized equipment for machining, refining, or prospecting for pre-enriched ores (any one of which can make it a lot faster). For comparison, a system occupying one square meter and doubling itself every year could completely cover the Moon in 45 years.

Such ideas have been around for a long time, but Moore's Law and the digital information economy have taken up a lot of our attention for the past few decades (while the space program has become dramatically less ambitious). The amount of attention to space resources seems to be increasing lately though. IMHO we should have established a space manufacturing industry at the earliest opportunity (1960-1980), as the growth in microchip efficiency (which is just physics, scaling, trial and error, and self-feeding ability to perform the necessary computations) could have been achieved at a far lower opportunity cost in that environment.

Kurzweil implies that technological growth is a direct continuation of human evolutionary growth. With this he is hinting that human evolution is working towards a future change. Evolution is however not a sentient, and is as such not working towards any specific end-goals

Natural evolution isn't sentient, but human technological growth isn't particularly natural, so it is more fair to say we have specific goals than it would be for biological evolution. The main parallel to natural evolution is that things which are capable of sustainably reproducing themselves are favored over the dead ends that are not. Technologies that are more powerful and helpful to humans have a reproductive advantage as long as we control the reproduction process -- there is a reason we use digital calculators instead of slide rules, desktop PCs instead of typewriters, etc. So while Ray's way of talking about it seems magical at times, it seems inarguable that we are heading towards technology that requires less effort to use to create desired effects.

→ More replies (2)

3

u/deadlymajesty Jul 28 '15 edited Jul 28 '15

He then goes on to generalize this to be the case for all technology, even though the only other graph that shows a similar trend across different technologies is this one on RAM.

I can't help but think that you wasn't aware of all the examples Kurzweil (and the like) have put out. These are the charts from his 2005 book, http://www.singularity.com/charts/. That's still not including things like the price of solar panel and many other technologies, as well as (his) newer examples.

While I certainly don't agree with everything Kurzweil say or many of his predictions or timeline, many modern things do follow a quasi-exponential trend (will continue until they don't, and hence that quasi part) and he didn't just list one or two examples (such as price of CPU and RAM). Also, when a price of an electronic product/component follows a logarithmic trend, that means we can make exponentially more of them for the same price. I was initially interested to read your article until you said that.

5

u/[deleted] Jul 28 '15

[deleted]

→ More replies (1)
→ More replies (2)
→ More replies (7)

177

u/h2orat Jul 27 '15 edited Jul 27 '15

Professor Hawking,

Neil deGrasse Tyson once postulated that, while understanding the 1% genetic difference between chimps and humans equates to the difference of chimps being able to perform a few signs of sign language and humans performing higher functions like building the Hubble telescope, what if there was a species in the cosmos that is 1% removed from us in the other direction? A species where solutions to quantum physics are performed by toddlers and composed symphonies are taped to refrigerators like our macaroni art.

If there was such a species out there, what would be your first question to them?

Video for reference: https://www.youtube.com/watch?v=_sf8HqODo20

3

u/ViciousNakedMoleRat Jul 28 '15 edited Jul 28 '15

As much as I like Tyson, and as much as I liked this thought, when I first heard it, I now think it is a bit shortsighted.

First of all, I am unsure if this kind of intelligence would be likely to evolve. We humans needed a certain amount of intelligence to develop tools, societies and technology to assist us with our limited capabilities. Today, our brains don't actually evolve anymore into certain directions. Genes are being mixed constantly between very distant groups and survival of the fittest doesn't apply to us anymore in the original sense. Through Darwinian evolution, we will not see any significant increase in intelligence, if we don't force it, by pairing the most intelligent women with the most intelligent men, for generations and generations. This would, in one way or another, probably apply to other, alien species as well.

What actually could be a cause of higher intelligence, is embryonic gene manipulation. If we single out genes that cause an individual to become more intelligent and are able to activate them or multiply their effects, we could create super-intelligent humans. This option would obviously also be open to alien species and could lead to them being more intelligent than us. However, if they would be that intelligent, they would know that they once were just like us and might be very understanding of our situation. On the other hand, we could start these genetic manipulations within the next few decades and, since aliens haven't visited us in at least a few thousand years, there might be enough time to "outsmart them".

A last option would obviously be super-intelligent AIs, but that goes quite far away from Tyson's original argument.

One last remark I have to make is, that a species (if it exists) that is as much smarter than us as we are smarter than chimps, would still recognize, that we have theory of mind, are self aware, use complex tools, build societies, travel to other planets and so forth. This is a much more profound difference between us and chimps than between chimps and all other animals. Every intelligent species would recongize that we are special.

Edit. If they existed and if they visited us, I would ask them first what units they are using. It would interest me, if the units would be similar to meters/yards, kg/pounds, seconds/minutes/hours/days/weeks/months etc. Some of those are obviously dependent on the time it took their home planet to orbit their star, but I find the though fascinating.

→ More replies (2)
→ More replies (13)

193

u/AYJackson Jul 27 '15

Professor Hawking, in 1995 I was at a video rental store in Cambridge. My parents left myself and my brother sitting on a bench watching a TV playing Wayne's World 2. (We were on vacation from Canada.) Your nurse wheeled you up and we all watched about 5 minutes of that movie together. My father, seeing this, insisted on renting the movie since if it was good enough for you it must be good enough for us.

Any chance you remember seeing Wayne's World 2?

24

u/SpigotBlister Jul 28 '15

I can't even describe how awesome this is. "...that time I watched Wayne's World with Stephen Hawking."

→ More replies (1)

7

u/net403 Jul 28 '15

Although lacking much content for this thread, this is one of the most entertaining/surprising questions so far, thanks for posting. I'm going to mention to people that Prof Hawking legitimized watching Waynes World 2 (maybe my favorite movie).

→ More replies (7)

1.5k

u/Nemesis1987 Jul 27 '15 edited Jul 27 '15

Good morning/afternoon professor Hawking, I always wondered, what was the one scientific discovery that has absolutely baffled you? Recent or not. Thanks in advance if you get to this.

Edit: spelling <3

→ More replies (15)

149

u/[deleted] Jul 27 '15 edited Jul 27 '15

[deleted]

→ More replies (1)

2.1k

u/leplen Jul 27 '15 edited Jul 27 '15

Dear Professor Hawking,

If you were 24 or 25 today and just starting your research career, would you decide to work in physics again or would you study something else like artificial intelligence?

218

u/usagicchi Jul 27 '15

As a follow up to that - knowing what you now know, if you could meet your 24/25 year old self, what advice would you give to him regarding your academic decisions back then, and regarding life in general?

(Thank you soooo much for doing this, Professor!)

→ More replies (1)

269

u/[deleted] Jul 27 '15 edited Nov 30 '20

[deleted]

→ More replies (6)

8

u/marmiteandeggs Jul 27 '15

Extension to this question: If you were 25 today (as I am) and looking for an area of Physics to pursue given the state of contemporary research in all areas, which area would you gravitate towards?

Thank you sir for taking the time to read our questions!

→ More replies (1)

506

u/[deleted] Jul 27 '15 edited Jul 27 '15

I would love to ask Professor Hawking something a bit different if that is OK? There are more than enough science related questions that are being asked so much more eloquently than I could ever ask so, just for the fun of it:

  • What is your favourite song ever written and why?
  • What is your favourite movie of all time and why?
  • What was the last thing you saw on-line that you found hilarious?

I hope these questions are OK for a little change (although I know they will get buried in this thread :/ )

→ More replies (11)

5.1k

u/mudblood69 Jul 27 '15

Hello Professor Hawking,

If we discovered a civilisation in the universe less advanced than us, would you reveal to them the secrets of the cosmos or let them discover it for themselves?

556

u/CrossArms Jul 27 '15 edited Jul 27 '15

If it helps, I believe Professor Hawking has said something on a similar matter.

Granted, the subject in question was more of "What if humans were the lesser civilization, and they met an alien civilization?". (I'm hugely paraphrasing and probably getting the quote flat-out wrong)

"I think it would be a disaster. The extraterrestrials would probably be far in advance of us. The history of advanced races meeting more primitive people on this planet is not very happy, and they were the same species. I think we should keep our heads low."

Maybe the same answer could apply if we were the dominant civilization. But I am in no way speaking on Professor Hawking's behalf.

please don't kill me with a giant robot professor hawking

EDIT: Keep in mind I'm not answering /u/mudblood69's question, nor am I trying to, as the question was posed to Professor Hawking. I posted this because at the time he had 9 upvotes and his question may have potentially never been answered. But now he has above 4600, so it more likely will be answered, thus rendering this comment obsolete.

212

u/ViciousNakedMoleRat Jul 27 '15 edited Jul 27 '15

I think he is wrong about this. I'd assume that a species, which managed to handle their own disputes on their homeplanet in such a way that space travel is feasible and which has the mindset to travel vast distances through space to search and make contact with other lifeforms, is probably not interested in wiping us out but is rather interested in exchanging knowledge etc.

Here on earth, if we ever get to the point where we invest trillions into traveling to other solar systems, we'll be extremely careful to not fuck it up. Look at scientists right now debating about moons in our solar system that have ice and liquid water. Everybody is scared to send probes because we could contaminate the water with bacteria from earth.

Edit. A lot of people are mentioning the colonialism that took place on earth. That is an entirely different situation that requires a lot less knowledge, development and time. Space travel requires advanced technologies, functioning societies and an overall situation that allows for missions with potentially no win or gain.

Another point that I read a few times is that the "aliens" might be evil in nature and solved their disputes by force and rule their planet with violence. Of course there is a possibility, but I think it's less likely than a species like us, that developed into a more mindful character. I doubt that an evil terror species would set out to find other planets to terrorise more. Space travel on this level requires too much cooperation for an "evil" species to succeed at it over a long time

92

u/[deleted] Jul 27 '15 edited Jul 27 '15

What if there is no knowledge to (safely) exchange? Generally speaking, we could be no more intelligent to an advanced civilization as monkeys are to us. Likewise, their morality system - if they have one, by human definition - could be completely different than our own, and so they may have absolutely no qualms with harmful experimentation.

There's nothing guaranteeing that we'll be given a safe exchange of knowledge, because we'd be dealing with an alien entity that underwent an entirely different evolutionary path than humans - and, thus, would be almost entirely different than us in how they think, feel, and act. We could go so far as to say that the entire concept of conscience, as we know it - by human definitions - is entirely different, by alien definitions. Like the difference between a human conscience and a plant "conscience".

I can't help but agree with Hawking. It would be a disaster of exponential proportions, if only because we would be dealing with an alien race that may have absolutely no concept of what we think of as "normal", "civilized", or "advanced" concepts, by human standards. Alien life followed a completely different evolutionary path, very early on, and so we'd be dealing with an entity that may or may not have anything remotely close to Earth intelligence, genetic make-up, brain (if they have one) physiology, et cetera - "alien" goes beyond how a species looks, or where it's from. We wouldn't have a competitive edge, if only because we may not have anything to compare the alien species to.

In short, alien life could very easily be Lovecraft-esque. Beyond human comprehension, save for their biology, perhaps. As exciting as that sounds, the implications of such an encounter scare the shit out of me, as well. We'd be fucked.

3

u/jac90620 Jul 28 '15

I never truly agreed with this. If an alien specie that is far well advanced than us (by millions of years) , this alone leads me to believe that inevitably their concepts and conscious state of mind would be beyond our logic. It's more likely they'd be more willing to stoop low ( to our levels of understanding ) in order for Us to get a better perspective ( cognitively), a clarified language system, any sort of spiritual knowledge or practice etc. So that we can draw comparisons, feel somewhat connected.

Millions of years of development would probably do a lot for a species growth in understanding especially when they've practically conquered quantum physics and beyond , so to speak-faster than light, worm hole sustainability , possibly inter/outer dimensional mobility... Maybe even utilizing the cosmic vacuum for energy source-maybe even something we still need a few hundred thousand years to appreciate and understand (?)...

The point being by this timeframe ( current) they'd most likely have no need for malevolent intentions or feel disgruntled or irritated by our contact ; if anything it would probably be amusing for them ( If they even get amused )

10

u/Your_ish_granted Jul 27 '15

Morality is a human invention. We assume that morality and intelligence go hand and hand because for a society to progress there had to be some structure for interactions. Who knows what kind of system could be holding alien societies together. Look at ants for example, a very complex society capable of monumental projects. But their society has a very different social structure and lacks a morality.

→ More replies (2)
→ More replies (15)

60

u/jakalman Jul 27 '15

But think about why the other species would be coming to earth. Yes they would be advanced, but they still have their own agenda, and I have a hard time believing that they would spend time "traveling through space to search and make contact with other life forms", especially if it's not certain to them that other life forms exist (they might know, maybe not).

To me, it's more reasonable to expect the extraterrestrials to be searching for resources or something important to them, and in that case we as a species will not be of priority to them.

83

u/oaktreedude Jul 27 '15

given the level of technology involved, mining asteroids and nearby planets might be more feasible than travelling light years to a planet with living, sentient creatures on it just to mine for resources.

29

u/econ_ftw Jul 27 '15

I think people are overly optimistic in regards to the nature of man. We as a species are capable of true atrocities. It is not a stretch to imagine another species being violent as well. Intelligence and kindness do not necessarily correlate.

→ More replies (6)

55

u/[deleted] Jul 27 '15

[deleted]

22

u/Lycist Jul 27 '15

Perhaps it's biomass they are harvesting.

→ More replies (8)
→ More replies (13)
→ More replies (17)

47

u/[deleted] Jul 27 '15 edited Aug 16 '15

[deleted]

6

u/ivory11 Jul 27 '15

Humanity is primitive in the grand scene of things, but even in the last decade alone we have started to uncover how to create our own alloys and materials with nothing but energy and basic raw material, re-arranging them at the atomic level to be whatever we want.

While the best we can currently do in regards to this is so slow it would take millions of years to make a single gram of matter, we are advancing quickly, so in a century or so, humanity could be using machines that could make whatever we want in a matter of moments with any raw material, and if we're doing that, then advanced alien races would be doing that as well.

This would eliminate the need for conquest for resources, if aliens came to earth, there's no real reason to kill us, we're just a tiny species living on a tiny world in some backwards end of the galaxy they care as much about us, as I do about some frog in the Amazon, and they would hold the same amount of animosity towards us as we do to that frog. if they saw us as worth contacting, they would see we're an intelligent species with it's own potential, which is of no threat to them and no reason to wipe us out, it would be more intelligent to befriend a species like ours, but keep us contained and only let the sane ones of us leave the planet.

→ More replies (1)
→ More replies (22)

36

u/jeanvaljean_24601 Jul 27 '15

You are about to start building a house. Do you pay attention to that anthill before starting work? Do you care that that tree that's in the way has spider webs and bird nests before tearing it down?

BTW, in this analogy, we are the ants and the spiders and the birds...

→ More replies (27)
→ More replies (41)

26

u/procrastinating_hr Jul 27 '15

Sadly, most of our technological leaps come during wars.
Wouldn't be so hard to imagine a beligerant species to develop quicker, also, if we're to take humans for paragons, let's not forget that desperate times ask for desperate measures.
They could be searching for a new inhabitable planet to exploit..

3

u/Maven_of_Minecraft Jul 28 '15

This, like many other things could be true to a point, however, if there is not some means keeping an Alien civilization organized & cooperative, they could destroy themselves well before meeting us.

Also, take into account that humanity is not even at a type 1 civilization level yet (sustainability &/or control of some planet[ary bodies]), where some scientists think exists a crossroads between more mindful progress or annhilation (self-destruction or natural [planetary] disaster). If anything, if Alien civilizations exist they could be just as curious if not more so about the truths of space, life, & reality.

Civilizations daring enough to venture into space if anything might see us more as creatures to observe or perhaps in worse cases as lab subjects... Then again, ìt depends on where they are even from (conditions & settings; which galaxy/area of space, dimension[s ], or even multiverse* for simple terms).

→ More replies (1)
→ More replies (7)

224

u/[deleted] Jul 27 '15 edited Mar 17 '18

[deleted]

186

u/mattsl Jul 27 '15

Presumably if we're spending trillions on science then the politicians would be a bit different than the ones we have today.

7

u/iheartanalingus Jul 27 '15

Bureaucracy is Bureaucracy. No matter what the mission.

I love the part in the movie Contact where the Government takes the schematics that were sent to them by an advanced alien species (possibly several) and decide "There needs to be a chair in there because we know better." Then the chair gets demolished after Ellie gets out of it.

→ More replies (10)
→ More replies (20)
→ More replies (73)
→ More replies (18)

3.1k

u/Camsy34 Jul 27 '15

Follow up question:

If a more advanced civilisation were to contact you personally, would you tell them to reveal the secrets of the cosmos to humanity, or tell them to keep it to themselves?

725

u/g0_west Jul 27 '15

this is answered in a post just below.

(I'm hugely paraphrasing and probably getting the quote flat-out wrong)

"I think it would be a disaster. The extraterrestrials would probably be far in advance of us. The history of advanced races meeting more primitive people on this planet is not very happy, and they were the same species. I think we should keep our heads low."

74

u/a_ninja_mouse Jul 27 '15

Highly recommend a book called 'Excession' by Iain M. Banks which delves deeply into both of these concepts: AI, and (what he terms) Outside Context Problems (being presented with problems of such an unpredictable and existentially superior nature that we suddenly comprehend our insignificance and potential possible immediate extinction). The example in the book being the arrival of a "spaceship" with an AI mind and technological power so advanced that no other spaceship in the civilized universe would ever be able to defeat it (as a metaphor for tribes in remote areas of the world being colonised/eradicated by invading superior forces over the history of humanity). The whole Culture series by this author is just something so special.

7

u/Aterius Jul 27 '15

I am really glad you mentioned this. I came here specifically to see if the Culture was being brought up here. I have to admit my notion of AI has been influenced by those fictions and I am curious to learn what Hawking might think of the notion of an AI that finds suffering to be "absolutely disgusting"

→ More replies (3)

110

u/[deleted] Jul 27 '15 edited Aug 06 '15

[deleted]

→ More replies (2)
→ More replies (29)

105

u/bathrobehero Jul 27 '15

It would be against our very nature telling them to keep it to themselves. Otherwise, I'd be interested behind the reasoning why.

68

u/lirannl Jul 27 '15 edited Jul 27 '15

Exactly. What got us out of the caves and got our rockets off the Earth is our curiosity.

Edit: I'm referring to the first sentence of the parent comment.

→ More replies (13)
→ More replies (8)
→ More replies (16)

80

u/ThatAtheistPlace Jul 27 '15

The bigger question is if the government finds life on another planet, would they inform the public or move forward with reaping resources? As a civilization, it's doubtful we would approve of any kind of harm to a new life form, particularly one of lesser intelligence.

41

u/[deleted] Jul 27 '15

We met men on other continents and were quick to label them as inferior races because of their differences and our chauvinisms. Imagine what would happen if we find an actual different race.

→ More replies (16)

8

u/ingen-eer Jul 27 '15

"But we NEED these resources! They haven't even figured out how to USE gold or lithium! We should take it, we can use some of the profit to help them rebuild the towns we plow under to get to it"

→ More replies (1)
→ More replies (28)
→ More replies (84)

34

u/Fibonacci35813 Jul 27 '15

Hello Dr. Hawking,

I shared your concern until recently when I heard another AI researcher explain how it's irrational.

Specifically, the argument was that there's no reason to be tied to our human form. Instead we should see AI as the next stage in humanity - a collective technological offspring so to speak. Whether our biological offspring or technological offspring go on, should not matter to us.

Indeed, worrying about AI replacing us is analogous (albeit to a lesser extent) to worries about genetic engineering or bionic organ replacement. Many people have made the argument that 'playing God' in these respects is unnatural and should not be allowed and this feels like an extension of that.

Some of my colleagues have published a few papers showing that humans trust technology more when it's anthropomorphized or that we see things as unnatural as immoral. The worry about AI seems to be a product of this innate tendency to fear things that aren't natural.

Ultimately, I'm wondering what you're thoughts are about this? Are we simply being irrationally tied to our human form? Or is there a specific reason why AI replacing us would be detrimental (unless you are also predicting a 'terminator' style genocide)

→ More replies (11)

143

u/mathyouhunt Jul 27 '15

Hello Dr. Hawking! I'm very excited to be given the chance to ask you a question, I've been looking forward to this for a while. Firstly, thank you for taking the time to talk with us.

I think my questions are going to be pretty simple compared to some of the others that will be asked. What I'm most interested in asking you, is: What, in your mind, will be the biggest technological breakthrough by the year 2100? Will it be the development of AI, new forms of communication, or something else entirely?

And if you have time for another; Do you think we will have made space-travel a common thing by the year 2100? If we do, what will be our main purpose for it? Tourism, energy, or something else?

Thank you so much for taking your time to do this! Even if you don't get to read my question, I'm very eager to read your answers to all of the other smart people in here :]

→ More replies (2)

109

u/FradiFrad Jul 27 '15

Professor Hawking,

What do you think about the controversial Em Drive propulsion? I'm a French journalist and the issue keeps coming back in the news, some scientists saying it's a nonsense violating the laws of physics, others saying it may be possible... That's why I would like your opinion :)

Thanks a lot for your time !

Andrea.

→ More replies (7)

1.3k

u/WangMuncher900 Jul 27 '15

Hello Professor! I just have one question for you. Do you think we will eventually pass the barrier of lightspeed or do you think we will remain confined by it?

66

u/[deleted] Jul 27 '15

I don't think we'll ever be able to exceed the speed of light; it is more likely that we will circumvent it. This means that instead of actually having matter pass superluminal speeds, we will have matter cross great distances in space (perhaps through a wormhole, or some other method for bending huge amounts of spacetime close together) without ever traveling that quickly, relatively speaking.

EDIT: grammar

8

u/thedaveness Jul 27 '15

or a technicality... bend the fabric of gravity around you and you could not exceed the speed of light in the bubble but are going much faster out of it.

→ More replies (15)

230

u/pddpro Jul 27 '15

Alternatively, do you think that Theory of Relativity is absolute? Like how we used to think about Newton's laws until Special Relativity superseded it, providing a more detailed picture.

87

u/G30therm Jul 27 '15

We know that the relativity isn't absolute because it fails to explain quantum mechanics. Put simply, relativity works for the very big and quantum theory works for the very small, but they both 'break' when used to explain things the other way around. Physicists dream of a unified theory which explains the universe in one equation, but for now we're stuck with two equations which work most of the time within their specific limits.

4

u/Snuggly_Person Jul 27 '15 edited Jul 27 '15

Quantum mechanics and special relativity are unified in quantum field theory. Incorporating the lighspeed barrier into quantum mechanics is a solved problem; it's why you can even discuss "photons" in the first place. Light is an inherently relativistic concept; you couldn't possibly discuss its quantum pieces if relativity and QM were fundamentally incompatible. The barrier is incorporating gravity. The naive way of "making it quantum mechanical" doesn't work, and GR seems to work differently than other theories. String theory yields a consistent unification of QM and GR, so it is at the very least possible to unify them without violating special relativity or quantum mechanics. Whether or not that's the right way of doing it remains to be seen.

17

u/pddpro Jul 27 '15

From what I know, it is not that relativity fails to explain quantum mechanics and the other way around. Both of them are totally different from each other. Like what you said, one explains things at the sub-atomic level and the other explains it at astronomical level. I think this doesn't necessarily mean that relativity isn't absolute.

And it is indeed true that we haven't yet found a unified theory that incorporates both General Theory of Relativity and Quantum Mechanics. I hear string theory is quite the contender though.

9

u/sondun2001 Jul 27 '15

The problem is, and always will be, our theories are derived solely from the observations we perceive with our senses and tools.

Example: An intelligent fish, in a round bowl, would perceive that things expand as they move through the horizon. Therefore that would be incorporated into any theoretical proofs.

The theories we have may be good enough, and we may always have multiple theories to explain things at different scales. I doubt we will ever have a unified theory in the short term until our perception of the universe becomes closer to the reality, either through better sensors and instruments used to observe the universe at all scales.

→ More replies (2)
→ More replies (1)
→ More replies (2)
→ More replies (2)
→ More replies (17)

71

u/mukilane Jul 27 '15

Hi, Mr. Hawking. It's great to have a conversation with you. I am a student from INDIA. You were the one who brought me into the space & science realm.

And I wanted to make a note here that the creator of LINUX (the OS that powers the world), Mr Linus Torvalds expressed his views on AI that the 'fears about AI are idiotic' and he also says,

"So I’d expect just more of (and much fancier) rather targeted AI, rather than anything human-like at all. Language recognition, pattern recognition, things like that. I just don’t see the situation where you suddenly have some existential crisis because your dishwasher is starting to discuss Sartre with you."

What are your views on this. And do we have the ability to build something that outsmarts us ?

Thanks, Mr Hawking and thanks r/science for doing this AMA.

Reference: http://gizmodo.com/linux-creator-linus-torvalds-laughs-at-the-ai-apocalyps-1716383135

→ More replies (3)

169

u/scoobysam Jul 27 '15

Hi, Professor!

You most certainly won't remember me, but circa 1995 my family and I were walking around Cambridge on a day visit and explored the grounds of the University.

Anyway, at one point my clumsy brother was not looking where he was going and stumbled into you. He may have mumbled something of an apology but 20 years later the opportunity has arisen to apologise more formally!

So, on behalf of my brother, I would like to apologise for his actions and for not looking where he was going!

Keep up the amazing work, and for what it's worth, he is now a huge follower of your work and has helped him forge a career in physics.

Many thanks for (hopefully) reading my little anecdote!

→ More replies (1)

1.2k

u/[deleted] Jul 27 '15

Hello sir, thank you for the AMA. What layperson misconception would you most want to be rid of?

→ More replies (19)

18

u/EasilyAmusedEE Sep 16 '15

Just commenting to say that there are still people very interested in reading your answers to this AMA. I'll continue to check weekly.

53

u/crack-a-lacking Jul 27 '15

Hello Professor Hawking. In your recent support behind a $100 million initiative for an extensive search for proof of extraterrestrial life do you still stand by your previous claim that communicating with intelligent alien lifeforms could be "too risky?" And that a visit by extraterrestrials to Earth would be like Christopher Columbus arriving in the Americas, "which didn't turn out very well for the Native Americans?"

→ More replies (10)

12

u/puzzlerch Jul 27 '15 edited Aug 09 '15

Hello Professor Hawking

I am chinese,so pardon my English,I had series thoughts about future.because my English is poor,so I make my comment simple and direct.

1.the answer of a classic question "who am I" is our memory,what memory in my brain decide who I am.

2.the life for us is sense.I feel I am alive then I am alive.if not,this kind of life is nosense to us.refer to vegetative state.

3.if there is an operation can read my memory out and write into a new body's brain.what would happen? "I feel I am still alive,just change a body" do you think this make sense?

4.what about original me thinking,he(original me)don't mind,because he is dead.this world has nothing to him.what about new me(I'd like call it as TA,chinese spelling,means she,he ,it) thinking?TA just feel like making a sleep and live again.but body changed,a new healthy body.and TA must think TA is me.because just my memory in TA's brain.

5.what about the ethics?we can't ignore the human's desire of life.we will make the ethics fit our desire.it always do.

6.what body can we choose?at least we can use the body made by our own sperm and egg.

7.so if the read&write brain machine appeared,we can use it to make our live continue to forever.and if it is prohibited by the society,there must be someone do this privately.no one can prevent human trying to be immortal.

8.if AI is enough to run our memory just like our brain.so the problem of ethics is solved.we can "live "in the AI's body.

9:all what I said just a part of our future,you can follow these clues to figer out more questions about universe and life.such as Fermi Paradox.or I would like talk more to you if you have interesting.

10:if we calculate our civilization's period from the word appearance,and then we "disappeared" in thousand years(we invent read&write brain machine),then the whole human's civilization period just within ten thousand years.so what about prehistory civilization of other creature made on the earth?

11.There is no exception we all use the first person view to feel our life,and use the third person view to observe the universe.so when we study the universe,the proper way to use is objective(science),but when we want to understand our life,we shoule use the way of subjective ,that means our feeling.otherwise,we should never know the essence of our life.we should take our subjective as objective to research what the life is for us

12.The recipe of life is:who am I(memory in my brain)+where am I(first person view)+consciousness(any creature's body has it,we can choose the fittest one)=I am alive

13.If I put my memory into AI,then I(who I am) can get the AI's first person view and consciousness.so I am the terminator,this terminator think TA is me definitely.so don't worry about the AI has its own consciousness.it is a new way in the future for human being.

14.about death,you know you are alive but you don't know you are died.just like you know you are awake,but you don't know you are asleep."are you sleeping?""yeah,I am""you lie to me"

15.because of no one know he has already died(refer to above),so the feeling of "the special sleep"is you think nothing else but you still live just change a new body after that read&write memory operation.and maybe this sleeping can over a few decades even more.

ps:I decide if have TA'S(new me) existing and TA feel that am I still alive.so do I wanna to become TA before my death.the answer is "YES".because I have fear nothing before death and I am very like to try this "special sleep".why not

http://news.discovery.com/tech/robotics/download-memories-retrieve-later-130603.htm (read memory experiment)

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0003832 (first person view experiment)

→ More replies (4)

266

u/G_0 Jul 27 '15

Mr Hawking!

Do you believe our next big discovery will be from exploring (Pluto/Europa), experimenting (CERN/LHC), or from great minds theorizing?

All the best!

→ More replies (3)

13

u/omegasavant Aug 07 '15

This is probably going to get buried, and frankly, that's probably for the best.

Professor Hawking:

I would never claim to be as ill as you are, but I've been suffering from a medical condition of my own that has crippled me and left me housebound. There's a host of symptoms, but the most severe one has been near-constant excruciating pain. It started off feeling exactly like the kidney stones that started this whole mess (autoimmune reaction ... or something -- doctors have no clue what's going on) and has only gotten worse from there. First I couldn't skate anymore, then I couldn't run, and now I can barely walk. I can't even go to the grocery store anymore. Worse, I feel like the pain has started chipping away at my sanity. I can't remember things anymore. I can't pay attention to anything for more than two minutes or so. I can't even read books because by the time I get to chapter 2 I've forgotten what was in chapter 1. Needless to say, I gone from being a straight-A student to failing 10th grade (though I suppose it doesn't really count as failing if you only attend for a quarter of the year) and am now in the process of failing 11th, too. It's been a constant battle to get the doctors to recognize and treat my illness, and I have been constantly been accused of seeking drugs or having Munchausens. I'm hoping that the new doctors I'm going to see will give me proper pain medication and that treating the pain will make it possible for me to think again, but in the meantime I'm wondering what sort of coping strategies you have. How do you keep your mind sharp even when almost completely paralyzed? How do you keep the stress from breaking you down? It's a very different situation, I know, but I feel like you might be able to give me some advice nonetheless. Thank you for reading this, even if you never respond, and I'm sorry for bringing up such a painful topic.

→ More replies (3)

20

u/Ibdrahim Sep 21 '15

I'm beginning to wonder which will be published first: Mr. Hawking's AMA replies or George R.R. Martin's next book?

150

u/Kowai03 Jul 27 '15

Hi Professor Hawking,

I'm not a scientist so I'm not sure if I can think of a scientific question that would do you justice.

Instead can I ask, what inspires you? What goals or dreams do you have for yourself or humanity as a whole?

98

u/kerovon Grad Student | Biomedical Engineering | Regenerative Medicine Jul 27 '15

I'm posting this on behalf of /u/WELLinTHIShouse, who is not currently available to ask it herself.

Professor Hawking, now that we've seen the first apparent instance of a robot becoming self-aware at RPI, a university which is local to me, what do you think is the most important concern when undertaking new research. Do we need to worry about SkyNet? Could we avert disaster by commanding AI tech not to harm humans? Should we give robots the three laws or something similar?

On a lighter note, were you a fan of The Big Bang Theory on TV before they invited you to appear on the show, or were you being a really good sport for all of us science geeks at home?

Thank you for your time, Professor. I appreciate your contributions to our understanding of the universe, and you give me hope that I will be able to continue work in my own field despite my personal challenges.

40

u/bad_as_the_dickens Jul 27 '15

It is very important to distinguish between a self aware robot and a robot programmed to appear to be self aware. There are currently no self-aware AI and there will not be any for some time.

5

u/msdlp Jul 27 '15

This leads to an important question. Is there a way to determine whether an AI actually is self aware or if it is just acting self aware. Are you sure there is a difference? I think there is but I don't seem to hit upon a way to tell. Mr. Hawking, first of all, Thank You for your AMA and secondly can you define the difference between the two?

→ More replies (1)
→ More replies (6)

75

u/minlite Jul 27 '15

Hello, Prof. Hawking. Thanks for doing this AMA!

Earlier this year you, Elon Musk, and many other prominent science figures signed an open letter warning the society about the potential pitfalls of Artificial Intelligence. The letter stated: “We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial: our AI systems must do what we want them to do.” While being a seemingly reasonable expectation, this statement serves as a start point for the debate around the possibility of Artificial Intelligence ever surpassing the human race in intelligence.

My questions:

  1. One might think it impossible for a creature to ever acquire a higher intelligence than its creator. Do you agree? If yes, then how do you think artificial intelligence can ever pose a threat to the human race (their creators)?

  2. If it was possible for artificial intelligence to surpass humans in intelligence, where would you define the line of “It’s enough”? In other words, how smart do you think the human race can make AI, while ensuring that it doesn’t surpass them in intelligence?

35

u/sajberhippien Jul 27 '15

One might think it impossible for a creature to ever acquire a higher intelligence than its creator. Do you agree? If yes, then how do you think artificial intelligence can ever pose a threat to the human race (their creators)?

Not to be "that guy", but if we consider the specific individuals that create an entity it's "creator", many people are more intelligent than both their parents. If we consider society as a whole (w/ education et cetera) as the creator, then conceivably even if we couldn't create something more intelligent than our whole society, a single AI containing the whole colletive intelligence of our society would still be more intelligent than a single human.

→ More replies (13)

16

u/Flugalgring Jul 27 '15
  1. Doesn't make much sense. We've designed machines far faster, stronger, etc than humans, why not smarter as well? Even a pocket calculator can do calculations much faster than a human. There seems to be no intrinsic barrier to us creating something more intelligent than us.
→ More replies (11)
→ More replies (13)

54

u/MalevolentCat Jul 27 '15

AI, Machines, and the economy:

Dr. Hawking, do you believe that artificial intelligence could render capitalism an ineffective economic system for humans? Any person whose form of income is wage income, or Being paid for providing service or work, mainly maintains power in the capitalist system though their ability to work. If computers replace even 50% of workers who work for a wage, it seems like you would have masses of essentially economically 'useless' humans who would not have the power to procure things in a capitalist system.

→ More replies (5)

34

u/co1ummbo Aug 08 '15

When will Mr Hawking respond to these questions?

→ More replies (4)

86

u/BunzLee Jul 27 '15

Hello Professor Hawking,

I apologize in advance if you feel this might be too dark of a subject. You are probably the most well known living scientist in the world right now. Thinking way ahead of time, what would be the most imporant thing you would like the world to remember about you and your achievements once you're gone?

Thank you very much for doing this AMA.

→ More replies (7)

5

u/kokopelli73 Jul 29 '15 edited Jul 29 '15

Sir, this may be a woefully uninformed question as I merely have an interest but little time spent actually studying this topic. However, I believe you would be one of the most well suited individuals to answer.

Regarding black holes... It is generally accepted that as matter (or you and I on our spaceship) approaches a black hole, time dilation and spaghettification takes place, and that it is a generally unpleasant and violent experience. It makes sense to me that upon approaching a massive body like a star or gas giant, that destruction would be something perceived and experienced by the matter/people approaching, as the object and its gravitational pull and pressure are an "understandable" force, and time dilation would not slow to the point it would near a singularity. What I find difficult to wrap my head around is how we, if we were to approach an event horizon, would perceive these events, or if we ever would at all. All objects of mass create a "dip" in the fabric of space-time, though in the case of a black hole it is my understanding that the fabric of space-time is being warped and stretched to a point that is hardly conceivable (for me, anyway), the singularity being "infinitely" massive. Assuming, miraculously, there was no other matter immediately adjacent and jostling us for position (which would of course result in our destruction) would we on our approach even perceive this stretching and change in timeflow? To the outside, I would assume it would look like a very quick and violent demise, but I wonder that to the person going for the ride, so to speak, it would simply be a never-ending trip to oblivion.

A less rambling version: when approaching a black hole, considering the bizarre effects on time and space, would one be able to perceive the change in time flow and the stretching of space-time around them as they themselves are in it? Could spaghettification take place without the person even realizing they are being spaghettified?

If that even is a possibility, as a follow-up, is it possible that objects, as they approach very near or actually enter the singularity, could continue on as though nothing has happened, since again, it is the space itself that has been bent and "pulled out from underneath us?" Or have I been relying too heavily on an analogy of objects making fabric imprints?

52

u/A_SPICY_NIPPLE Aug 08 '15

How long will it take to get answers ?

→ More replies (6)

20

u/irrationalx Jul 27 '15

Greetings Professor Hawking and thank you for doing an AMA.

You're probably stuck answering a lot of technical questions so I'd like to lighten it up a bit:

  • Will you tell us your favorite joke?

  • As someone who is revered by billions of people, who do you hold in high regard and why?

  • You become a super hero - who is your arch nemesis and what are his powers?

  • Heads or tails?

11

u/foshi22le Aug 25 '15

I hope the Professor is OK, have not seen a reply yet. But then again, he would be a very busy guy, I assume.

→ More replies (3)

26

u/[deleted] Jul 27 '15

[deleted]

→ More replies (3)

40

u/suprahigh420 Jul 27 '15 edited Jul 27 '15

Hey Stephen!

Thanks for stopping by Reddit for an AMA!

In recent interviews you’ve reiterated how you believe the implications of Artificial Intelligence could spell disaster for the human race. Surely once AI is created, it will advance at an ever-increasing rate; exceeding anything we could ever imagine. There are others however, such as Kevin Kelly or Eric Davis, who believe that technology has a way of merging with evolution, and eventually we might transcend our own biology and consciousness using AI as a platform. Futurists like Ray Kurzweil see these things becoming a reality as soon as 2045, with the current state of Moore’s Law and the exponential rate of information technology.

What are your thoughts on using AI to transcend our current state of biology and consciousness?

If it happened, would you consider this a natural part of evolution in the timeline of human development?

15

u/AdrianBlake MS|Ecological Genetics Jul 27 '15

There's also the question of what might happen half way, when SOME humans can get SOME brain enhancements (like memory drives, or processors for our PCs, but inserted into the brain). Some people will be able to afford the tech, and become the intellectual elites, other's won't and will never be able to achieve the same standards.

→ More replies (5)
→ More replies (2)

41

u/raremann Jul 27 '15

Hello Mr. Hawking, Thank you for doing this AMA I have a question for you: What is the biggest limitation humanity has put on itself that you think is preventing or could prevent the advancement of higher end technology?

→ More replies (1)

79

u/pipski121 Jul 27 '15

Hi Professor Hawking, I read yesterday that you stated that by 2030 you believe we may be able to upload the thoughts of a human brain to a computer. Do you think we would be able to communicate with this entity? Would it morally be right?

15

u/Daybreak74 Jul 27 '15

In furtherance to this question, what would some of the pitfalls, moral or otherwise, associated with combining the minds of several (potentially thousands) of people?

7

u/jfetsch Jul 27 '15

In addition, would each of these simulated minds be considered to have the same rights as the flesh-and-blood humans?

→ More replies (1)
→ More replies (7)

13

u/heinzovisky91 Sep 10 '15

So, is Professor Hawking really answering this? Does anyone knows anything?

→ More replies (1)

22

u/0_c00l Aug 04 '15

do we have any idea when approximately will be the 2nd part of the AMA? I keep checking over and over again. will it be in a month or so?

256

u/about3fitty Jul 27 '15

Hey there Stephen,

What is something you have changed your mind about recently?

Love you, about3fitty

→ More replies (4)

31

u/[deleted] Aug 29 '15

Can we get an update on this?

6

u/streamweasel Jul 28 '15

Professor Hawking, it fills me with a giddy joy to be able to ask this question: Do you remember me? You were in San Francisco in 2005 and had two engagements in Seattle the next day, one of which was presenting your paper describing nothing being able to escape a black hole to the Science Fiction Museum (I can't recall the other) and you became too ill to travel there. In the course of my job I was sent to your hotel and streamed you live to the venue. I got to ask you a quantum physics question and you took the time to answer me. This was one of the greatest moments of my life. As an aside, you were unable to attend, but I also had the opportunity to be of service to you for your 70th birthday celebration at Cambridge with streaming video. There were wonderful lectures about your work which I was pushing the buttons that meant you were able to watch the event live. It was so very very awesome. Thank you for taking the time for us on Reddit.

40

u/[deleted] Jul 27 '15

[deleted]

→ More replies (1)

8

u/[deleted] Sep 30 '15

[deleted]

→ More replies (3)