Sources of existential risk

Do you think humanity might face some kind of existential threat in the next two centuries?

Even if you think human extinction in the twenty-first century is unlikely -- I think it's really, really unlikely, I estimate about a 1% chance of it happening -- if human extinction is to occur, what do you think the most likely causes might be? Proposed causes include anthropogenic global warming, strong artificial intelligence, nuclear war, cybersecurity-related issues (e.g., destruction of critical infrastructure by cyberterrorism), biotechnology risk, and even "cosmic risk" (e.g., vacuum decay if the universe is in a false vacuum, Jupiter's gravitational pull making Mercury's orbit unstable).

As an extension of this -- to be a bit more specific -- what are your thoughts on strong AI research and the development of strong AI more generally?

My goodness! No one has the ability to accurately predict the world 200 years into the future. To envision a better perspective of how ludicrous your request is, consider someone from the early 1800s predicting the internet and digital storage, cars, aviation etc. There's just no way -- it's a wild guess. Alas, you'll have some of Dart's finest throw their guesses at you, thinking they're intelligent predictions with any merit.

Whilst I'm being hyper-generalised, I think AI development is the way forward, in terms of human evolution. In particular, any cyborg or transhumanist advancements, of which blunt/eradicate human flaws, should be welcomed. Furthermore, having AI to complete menial tasks is an excellent solution to the inevitable problem advanced civilisations encounter, when they become too comfortable (they import foreign people, of which eventually turn on the natives).

Haven't we 'been under the threat of nuclear war, since we created nukes?

Just because nobody lit the fuse,

Just because people stopped talking about it,

Doesn't mean it's not there.

I'm not sure nukes would kill all humans though.

. . .

I don't know enough about global warming to have a strong opinion,

Though I acknowledge local shifts in nature,

Like short term, the Dustbowl in Grapes of Wrath,

As well as some global shifts,

Such as rising sea levels,

Maybe bad for future Florida, X hundred years in future, worthwhile for them to act now, but they won't I imagine.

I don't think minor global shifts and local shifts would kill all humans,

But I don't know anything about 'big implication.

. . .

Cyber concerns are a nuisance, than a death threat,

We survived before electronics,

'Could occur a lot of deaths, wars.

. . .

I find GMOs and mass vaccines concerning,

I think it better to store humans in separate honeycombs, than to effect 'all humans the same way.

. . .

Of AI,

I'm far too uneducated and uninformed to have an opinion,

I might not notice they're a thing until they're right in my face.

But if everything is pointless any way.

Nonetheless, material evolution seems to be headed somewhere.

It might not matter if we're not along for the whole ride.

Perhaps our memories might be.

People vary on how much they plan for the future,

Certainly some, even if they won't be alive, like their works to survive,

Like to leave a footprint, an echo, a name.

Or hope that their efforts will prove to help 'something in the future,

Be it humans, tribe, nature,

More than their name.

. . .

Some people hold truth a value,

And strive to deeds and honor worthy,

Someone might clean their campsite,

That the next user, enjoy nature, not weary trash picking,

Duty.

Cleaning the campsite.

Is that metaphorical?

If the Earth was ridded of humanity, do you think that it would ever completely clean itself of every scrap of human detritus.

Or do you think that our legacy would exist until the end of the Earth.

Yes and no,

Something I remember from my childhood is camping,

And my Dad always made a point of leaving the campsite looking clean when we left.

But it also matches with general idea of leaving 'anything well, for the next person who uses it,

Wipe off gym equipment,

Pick up after dog on sidewalk,

Vote for eco friendly bills that prevent business from dumping waste in river that grandkids can't swim,

Or air that grandkids don't choke on pollution.

. . . 

I've heard that people used to think there were canals on Mars, that they thought evidence of life,

Of course now we know there is no life on Mars, supposedly,

I think there's not anyway.

Of human creations, I don't know 'how long they'd last,

I've heard before about pieces of trash lasting longer than other pieces of trash, paper compared to metal cans or glass bottles.

Surely some architecture would last longer than others,

Surely 'fossils of human works could exist, and there are some 'really old fossils of times before humans.

I couldn't say what will last to the end of the Earth,

End of the Earth 'sounds a long time away,

A time farther away than the end of 'life on the Earth,

What 'is the expected end to the Earth?

Black hole?

Exploded sun?

. . .

End of humans did cross my mind briefly as of late, on account of one of Avery's topics,

Sent my mind down a rabbit hole,

I don't really see why the 'world would be better without humans,

Animals still die, suffer, kill one another, even the plants.

Logically if humans were concerned about life, (In the sense of philosophers who think negatives outweigh positives)

Wouldn't they commit Ecocide - Wikipedia?

People argue abortions ought be preformed on fetuses with disabilities,

Doesn't an animal such as a cow have mental facilities far less than a human, short lifespan,

Is such a life 'worthwhile?

And as animals can't consent,

They ought be looked after and not taken advantage of,

Though meat is tasty,

And way to look after, would be to not allow any more of them to be born, if such a life is not seen as worthwhile.

'Weeiiiird rambling on my part here, unorganized,

But it has been on my mind last hour or two.

Exploded Sun probably in approx 5 billion years.

Do you think that we will have managed to escape?

Perhaps as frozen embryos on board an inter-galactic fridge.

Overseen by A.I. (Another intelligence)

Will we be able to trust A.I.

Will it deem us necessary.

As for your other dilemmas all that I can say is, that we only care because we can and do.

Otherwise, thoughtlessness.

And how would material evolution progress to a meaningful end without thoughtfulness?

See above.

Future technology and it's implications, I find beyond my imagination, how it will change society, humans,

I can imagine changes, but not 'certain changes,

I see coinflips, but not the side that falls face up.

I think AI can be trusted to follow it's programming,

As a brick can be trusted to hold a house,

Though eventually perhaps, properties required to continue function, or unprogrammed questions will pop up.

An AI in a video game, will at times be unable to progress, where a human could,

Such as running out of resources, yet still have troops, just mill about base.

Though,

Humans will at times will be unable to progress where an AI could.

. . .

Of nihilism, I'm trying not to think about it, right this second,

Otherwise I should scream,

Also I feel bad at times, about encouraging such thoughts in other people.

. . .

An iceberg doesn't need thoughts, to progress to water.

An iceberg is just frozen water, inanimate save for winds and currents.

And A.I. as in another intelligence is another intelligence that makes its own decisions, not requiring human input.

It would seem likely that sometime in the next few million years this will happen.

The inevitability of material progress, as I see it.

And let me remind you that nihilism is simply a philosophical concept by default.

Even suicide serves a purpose.

Though perhaps I shouldn't be saying such things.

Stay positive, even if it's just clearing the litter from the campsite.

What is more material than the inanimate?

I mentioned iceberg because you said,

Though, I suppose it 'would lack meaning,

If meaning can only be given by a conscious observer or intender.

. . .

Of True AI, I've yet to know,

Either in Strong AI or Conscious AI.

Wherein lies an iceberg's thought.

The iceberg's movement is only given meaning by the conscious observer or an intender.

Another angle to the latter A.I. dilemma, is downloading human consciousness.

Something I find quite a scary concept.

Consciousness without life at the behest of the on/off switch.

Goodnight and sweet dreams.

I don't know where consciousness, is.

. . .

Of downloading,

I find it insulting more than scary, myself,

Well maybe a bit scary, if one considers further implications and abnormal norms of it.

Depending on what you speak exactly.

One doesn't take a singular piece of information online, when downloading,

A copy is created.

Somehow I find the idea of a copy of myself, insulting,

Were I to meet myself, I worry it might end up as ,The One (2001)

Because of my irritation,

At seeing my uniqueness of self/me, defied.

. . .

Various magic systems and advanced science fiction,

I've found disturbing at times,

Because of various horror fiction,

Sure people can be brainwashed in science fiction, Total Recall (1990)

Or fantasy, (Title With-held)

But there's nothing stopping a lobotomy in real life, person being imprisoned and 're-educated.

It's just how 'easy it is, 'what it is, and the fiction's description designed to induce 'more horror, that horrifies,

Though again, not really 'good in itself anyway.

I'm rambling,

But I bring it up,

As the idea of a virtual world, where others have such enormous power, I find frightening,

Though I suppose I live in material world where government has power,

They have different power, different presentation, that makes it 'seem normal.

. . .

Don't humans have an off switch?

Download is just transformation, of material, not pattern, I'd think,

If one is able to avoid copying oneself, and it 'truly able to change ones location, from meat and electricity, to metal and electricity.

Well, I think consciousness is located/generated/is the brain,

So I imagine one might be able to Ship of Theseus the brain with metal parts, and perhaps remain the same consciousness,

But I've heard confusing arguments and theories about consciousness before, so I'm not certain.

This post is four years old, written when I was 16. Perhaps needless to say, my views on this issue have changed pretty significantly. 

As they will probably change again.

A thinking person's views will continually change.

I would suggest that there is a distinction to be made between thinking and unthinking people.

Not basic functionality, but the conscious and continuous effort to develop and modify thoughts.

Lone Wolves as opposed to the flock of Sheep.

Well, the human off switch is brain death.

And consciousness comes somewhere under the umbrella of electro chemical signalling within the human on-board computer.

Transferring those signals to an alternative computer, is surely feasible.

Would we be aware of an alternative existence outside of our original organic support mechanism?

After all, isn't consciousness, awareness of existence?

Does consciousness and awareness require anything other than an energy source?

@zedvictor4

Hm, I feel like I've thought about consciousness in the past, but I've forgotten of what I thought.

Doesn't 'anything have an energy source?

Even rocks,

But a person braindead vegetable, is not 'thinking. . .

Animals respond to stimuli, have brains, neurons, memories,

Motion sensors respond to stimuli,

But really 'anything responds to stimuli, even rocks are effected by hammers.

. . .

Some people argue that fetus, does not have consciousness, awareness,

Do they?

Or do they mean human intelligence?

All these questions are rhetorical, and directed at me really.

. . .

Mental health patients, often say they don't 'feel like themselves on medications,

Even if when off medications they're manic, or have schizophrenia,

Hm, I imagine consciousness requires complex code, patterns, material,

DNA is vast, took long to become as we are now,

Brain effected by even vitamins, or lack of, lot in body effects quality of the brain,

'Human intelligence, I'm doubtful we can reproduce anytime soon.

@Tejretics

Ooh, say on, say on?

I would suggest that there is a difference to be made between effect/affect and responding to a stimulus.

The hammer produces an effect, and the rock is affected.

But the rock doesn't go ouch.

But of course, certain rocks contain energy sources.

Material reactions that produce an effect without thinking.

And for sure organic masses are an amazing outcome of material development.

And high maintenance with it.

And so, all subject to both the limiting factors principle and potential toxicity. Whereby cells will respond at a sub-conscious level, and we will function correspondingly.

And then we have pathogens to deal with.

And a foetus will become aware relative to its physiological and sensory development and also its environment.

What are your first memories? As in stored data that you are able to recall.

Some people will say that they can remember being born, but I doubt that they're being honest.

I have no clear memories of anything before the age of five.

Perhaps that's because I had nothing to fear until I was made to go to school and interact with the wider world.

Random thoughts really.

I would 'say, I have memories before five years old, though 'what age I couldn't say,

As I remember living somewhere else, before kindergarten,

But it's hard to call them memories, as the 'true pictures in my mind have been forgotten,

Instead I feel as though I have memories that have been transcribed and copied multiple times,

That 'exact truths of them, are doubtful.

I remember being in a bathtub, with a bath toy, plastic shapes of various shapes colors, that socketed into each other, and could be taken apart,

I remember throwing up, from being on a playground spinner/merry go round,

I remember one of those glass beads, you find in fishbowls sometimes, though I am unsure of it's color, I remember it as green.

But as I said earlier, the memories have become vague over time,

And I often suspect though I remember the 'words roughly, of my memory,

The 'memories are created much from the memory of my word repeated over time, of what was.

Humans ----via Meta-space mind/intellect/concepts--  create purpose aka to have a point or find that point.  Universe has no point/purpose.

Those who have not purpose/points are deemed or deem themselves as hopeless.

Biologic life seeks/desires and motivated by genetics to move in pointed direction. Some genetics lead to access of Meta-space mind/intellect/concepts and ego that ponders broadly on the direction, options, time, ,others, value etc.

Biologic life always make choices.  It is nature of complex consciousness. To consider, ponder, re-ponder, re-consider, and eventually act, or react etc.

I agree.

Obviously the biggest and most immediate threat to humanity, is humans, barring one or more of the 8 differrrent kinds of nasty volcanoes dont block out most of the sunlight for 2 or more years.

Humans appear smart on individual levels, but on collective level they are dumb and getting dumber, while having access to more accurate local and global data. The problem is ego. That is my best guess.

I do not understand how y’all have necroposted a thread from four years ago back into existence.

Sometimes I think a category has gone too many days without activity,

So I read some old threads,

And find my mind sparked at times, of opinions, curiosity.

It's certainly higher than 1%. The entire reason for the Fermi paradox is there is some very dangerous things that civilizations face. 

AI is an existential threat, and not just for the obvious reason it could replace us. Google the grey goo hypothesis for just one example. Nuclear threats exist, global warming. The sun could literally explode and kill us at any second. The biggest risks come from ourselves for the most part.

Outside of ourselves, AI and natural occurrences like killer asteroids, we have the fact that the reason the universe is quiet might be because of a hypothetical thing like a roaming civilization that seeks out and destroys any civilization just before or at the moment it reaches a technological singularity.

We are constantly having researchers try to reach out to alien civilizations in the off chance a signal hits one, but they aren't considering that there could be a very good reason other civilizations have STFU. That reason could be a serious existential planet destroying threat. 

The closer we get to a technological singularity, the closer we approach a planet destroying event. 

One answer to the Fermi paradox is that we are a simulated universe, which could mean that if we approach a technological singularity it could cause us to use so much of the simulations computer resources we kill ourselves.

The threat to the existence of intelligent life on this planet is very severe, very real and very likely. We should proceed cautiously. 

Commonly posed existential risks applicable to the next 200 years

1. AI/Singularity

2. Climate change

3. Nuclear apocalypse

4. Outside risks (i.e., aliens)

5. Disease, especially engineered

It's up for debate how much we can do about 2.

1. and 3. are easy in theory; don't build AIs or Nukes. Unfortunately international conflicts incentivize defection because of the advantages afforded by development of the same.

4. is similar to 1. By the time we realize it's a problem, there will likely be nothing we can do about them.

Others risks

nanobots: essentially the Grey goo hypothesis mentioned by Wylted, but it doesn't really require any intelligence behind it, so I count this separate from AI as it could happen independently.

other outside risks:

* type 1, one's we can potentially manage, such as a meteor crash course

* type 2, ones we can't, i.e nearby supernovas

Volcanoes, 5 kinds in varying degrees of serverity, and

.........in kinds of gaseus substances the release and other particles to block out sun