By Creator
- All you need for SAT Math Here!
Thanks for reading, this is a port of a study guide that I am using to study for the SAT. This should be done soon, but I don't have that much time.
- some formulas that are not directly given that may or may not help
i oofed on the other one
- Introduction to Effective Altruism
Effective altruism (EA) means using evidence and reason to take actions that help others as much as …
wiki
- Combining vectors
One of the most useful things we can do with vectors is to combine them!
- Elementary Algebra
How do we describe relations between different things? How can we figure out new true things from tr…
- Geometric algebra
A geometric algebra is a Clifford algebra over the reals which represents Euclidean geometry and man…
- Geometric product
#Motivation
want to incorporate rotors like $e^{\text{I}\theta}$ and scalars $n$ in the same system
…
- Geometric product: summary
Product for [multivectors multivectors].
Associative, left and right distributive. Non-commutative. …
- Geometry of vectors: direction
What rotation would it take to line up this vector to this one?
- Group orbit
When we have a group acting on a set, we are often interested in how the group acts on a particular …
- Locale
Topology - but right
- Quotient group
Given a group $G$ with operation $\bullet$ and a special kind of subgroup $N \leq G$ called the "no…
- Vector arithmetic
Vectors: what they are, and how to add and scale them.
- Big-O Notation
This notation describes asymptotic behavior of functions.
# O(x)
A function f is O(g(x)) if, for la…
- Ackermann function
The slowest-growing fast-growing function.
- Logical Induction (incomplete)
The theoretically ideal algorithm for bounded reasoning with lots of computational resources
- Logical Inductor Notation and Definitions
A handy guide to the blizzard of notation in the Logical Induction Paper
- Logical Uncertainty
The study of resource-bounded inference.
- Preemptive Learning
One of the basic theorems of logical inductors, used to derive many other theorems that rely on "buy low, sell high" strategies.
- ε-ROI Lemma
The fundamental lemma for analyzing logical inductor properties.
wiki
wiki
- Coliteracy
The coliteracy of two parties is the
wiki
- 2017 US GDP growth will be lower than in 2016
- 2017 will have no interesting progress with Gaza or peace negotiations in general
- 2017 will not have any major revolt (greater than or equal to Tiananmen Square) against Chinese Communist Party
- AI arms race
This part of the plan describes how to prevent and handle AI arms race.
- Alexei Andreev: Personal
Fun things Alexei Andreev likes to do.
- Alexei Test Group
- Alexei's Blog
Will come up with a better name later.
- Angela Merkel will be re-elected Chancellor of Germany in 2017
- Arbital
Arbital is the place for crowdsourced, intuitive math explanations.
- Arbital "parent" relationship
Parent-child relationship between pages implies a strong, inseparable connection.
- Arbital "requires" relationship
A page can require a requisite if the reader needs to have it before they are able to understand the page.
- Arbital "tag" relationship
Tags are a way to connect pages that share a common topic.
- Arbital "teaches" relationship
A page can teach a requisite when the user can acquire it by reading the page.
- Arbital Blog
Stay up to date on all things Arbital
- Arbital Labs
Landing page for the Arbital Labs domain.
- Arbital Markdown
All about Arbital's extended Markdown syntax.
- Arbital Markdown questionnaire
How to ask questions in Markdown.
- Arbital Redesign
What awesome things are in Arbital's future?
- Arbital as single conversational locus
Proposal and outline of Arbital's steps to create a single conversational locus.
- Arbital comment
A comment is a way for you to express your thoughts and opinions within the context of a page.
- Arbital community input
Do you have ideas about how to improve Arbital which you think the community should discuss?
- Arbital content license
What license does Arbital use for its content?
- Arbital domain
What is a domain? Why is it important?
- Arbital editor
How to use Arbital's page editor.
- Arbital editor buttons
What do all the buttons in the editor do?
- Arbital editor's manual
A quick overview of Arbital features relevant to editors.
- Arbital editor: Advanced
Advanced features of Arbital editor.
- Arbital editor: Basics
The basics of how to use the Arbital editor.
- Arbital features
Overview of all Arbital features.
- Arbital greenlink
What happens when you hover over an Arbital link?
- Arbital groups
What are groups? How can I create a new group?
- Arbital hidden text
How to hide text in Markdown behind a button.
- Arbital lens
A lens is a page that presents another page's content from a different angle.
- Arbital lens: TL;DR
Much shorter version of Arbital Lens page
- Arbital likes
What are likes? When should I use them? What happens when I like something?
- Arbital mark
What is a mark on Arbital? When is it created? Why is it important?
- Arbital members should state their beliefs clearly, even if they only hold them weakly.
Sometimes there is a tendency for people to state a weak version of their belief, if they don't beli…
- Arbital norms should encourage disagreement.
- Arbital norms should encourage nitpicking.
- Arbital page
The Arbital is a series of pages.
- Arbital page summaries
Because only one summary is not enough!
- Arbital page summaries Markdown syntax
How to create page summaries using Arbital's Markdown syntax.
- Arbital page: Basics
Explaining the basic features of an Arbital page.
- Arbital path
Arbital path is a linear sequence of pages tailored specifically to teach a given concept to a user.
- Arbital proposed project
Collecting all project proposals under this page.
- Arbital query
What is a query? How to create it? How to resolve it?
- Arbital requisites
To understand a thing you often need to understand some other things.
- Arbital should hide probability/approval votes until the user votes
This is to avoid a potential [-anchoring\_effect]. Also, presumably the user would be still be able …
- Arbital should not require any member to have particular beliefs, but should instead focus on enforcing specific rules of conduct.
- Arbital subscriptions
What's a subscription? How do you change it? What to expect?
- Arbital subscriptions: Maintenance
Subscribing to a page with intention of maintaining it.
- Arbital unlisted page
What do you call a page that's not part of any domain?
- Arbital: Google Maps for knowledge
Take your understanding from where it is to where it wants to be.
- Arbital: better blogging
What makes Arbital the choice blogging platform?
- Arbital: fixing online discussion
How can Arbital do better than existing discussion platforms?
- Arbital: information hub
How will Arbital help you keep up to date on any given subject?
- Arbital: learning from Wikipedia
How is Arbital different from Wikipedia?
- Assuming significant overhead in monitoring recipients of a microloan, it's more efficient to let them keep the money.
A claim about microfinance.
- Author's guide to Arbital
How to write intuitive, flexible content on Arbital.
- Author's guide to Arbital explanations
Requisite used for teaching authors about Arbital explanation features.
- Author's guide to Arbital: Advanced
Requisite used for teaching authors about advanced Arbital features.
- Author's guide to Arbital: Basics
Requisite used for teaching authors about basic Arbital features.
- Author's guide to processing feedback
Requisite used for teaching authors about Arbital feedback features.
- Bayes' Rule and its different forms
This is an arc that includes different ways to look at Bayes' Rule.
- Bayes' Rule and its implications
This is an arc that includes implications of the Bayes's Rule.
- Bayes' rule: Beginner's guide
Beginner's guide to learning about Bayes' rule.
- Bitcoin will end 2017 higher than $1000
- Bone Fracture
A bone fracture (sometimes abbreviated **FRX** or **Fx**, **F<sub>x</sub>**, or **#**) is a medical …
- By end of 2017 ISIS will control less territory than it did at the beginning of the year
- Calcitriol
Calcitriol, also called 1,25-dihydroxycholecalciferol or 1,25-dihydroxyvitamin D<sub>3</sub>, is the…
- Calcitriol in the Treatment of Prostate Cancer
Review
Link: http://ar.iiarjournals.org/content/26/4A/2647.full.pdf
From the abstract:
> "Calcitr…
- Calcium
Calcium is a chemical element with symbol **Ca** and atomic number 20. Calcium is a soft gray alkali…
- Calcium Plus Vitamin D Supplementation and the Risk of Breast Cancer
Randomized trial
Link: http://jnci.oxfordjournals.org/content/100/22/1581.long
From conclusions:
…
- Calcium plus Vitamin D Supplementation and the Risk of Colorectal Cancer
Randomized trial
Link: http://www.fp.ucalgary.ca/FMResidentSecure/Articles/Ca%20vit%20d%20and%20ca%…
- Cancer
Cancer, also known as a malignant tumor or malignant neoplasm, is a group of diseases involving abno…
- Cardiovascular Disease
Cardiovascular disease (CVD) is a class of diseases that involve the heart or blood vessels.Alexei A…
- Center for Applied Rationality
Also known as CFAR. It's a non-profit organization created to figure out best rationality practices to mitigate AI x-risk.
- Cholecalciferol
Cholecalciferol (toxiferol, vitamin D<sub>3</sub>) is one of the five forms of vitamin D. It is a se…
- Circles of discussion
- Comprehensive guide to Bayes' Rule
This is an arc that includes all Bayes content.
- Concept
Add this meta tag to pages which are concepts.
- Countability
Some infinities are bigger than others. Countable infinities are the smallest infinities.
- Creating a /learn/ link
What options are available when creating a /learn/ link?
- Crony belief
**Crony belief** is a concept originally introduced in Kevin Simler's post, "Crony Beliefs". It's us…
- Crony beliefs (from Melting Asphalt)
The original article that introduced and explained "merit beliefs" vs "crony beliefs" dichotomy.
- Decision Theory
Nothing here yet.]
Automatically generated page for "Decision Theory" group.
If you are the owner,…
- Displaying the list of fundraiser donors sorted by the donation date would help with the "wait and see" problem.
- Does solar exposure, as indicated by the non-melanoma skin cancers, protect from solid cancers: Vitamin D as a possible explanation
Correlational study
Link: http://www.ejcancer.com/article/S0959-8049%2807%2900324-3/abstract
From …
- Donald Trump remains President at the end of 2017
- Donald Trump’s approval rating at the end of 2017 is lower than fifty percent
- Donald Trump’s approval rating at the end of 2017 is lower than forty percent
- Donor lottery
An arrangement where a group of people pool their money and pick one person to give it away.
- Dow Jones will not end 2017 down by more than 10%
- Easier, better, faster, brighter
In which Arbital gets a makeover, shortcuts, quick feedback, and more!
- Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial
Randomised double blind controlled trial.
Link: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC150177/…
- Effective altruism outreach
Effective Altruism Outreach (EAO) is part of the Centre for Effective Altruism and is an organizatio…
- Feed meta tags
Children of this page are meta tags that are used to add a page to a feed.
- Fewer refugees admitted by Europe in 2017 than 2016
- Focusing
Focusing is a psychotherapeutic process developed by psychotherapist Eugene Gendlin
- France will not declare a plan to leave EU in 2017
- Fundraisers should have a threshold amount which, if not hit, results in a refund.
When starting a fundraiser, a nonprofit should declare a threshold amount. If the nonprofit doesn't …
- Germany will not declare a plan to leave EU
- GiveWell: a case study in effective altruism, part 1
> GiveWell has recently [written about](http://blog.givewell.org/2015/11/25/good-ventures-and-giving…
- Hiring
What's Omniment's protocol for hiring employees?
- How much of the discussion around AI safety should be public?
- Hub page
This tag is applied to pages which server the role of a "hub": the user starts there, goes off to learn more about the topic, and then comes back. This meta tag modifies the page's UI.
- I often wait to see how much other people will donate to a fundraiser before donating myself.
- ISIS will not continue to exist as a state entity in Iraq/Syria by end of 2017
- Iff
If and only if...
- In 2017 SSC will get fewer hits than in 2016
- In 2017 at least one SSC post will have more than 100,000 hits
- In 2017 there will be no major intifada in Israel (ie > 250 Israeli deaths, but not in Cast Lead style war)
- In 2017, Assad will remain President of Syria
- In 2017, EMDrive is launched into space and testing is successfully begun
- In 2017, Israel will not get in a large-scale war (ie >100 Israeli deaths) with any Arab state
- In 2017, North Korea’s government will survive the year without large civil war/revolt
- In 2017, Trump administration will not initiate extra prosecution of Hillary Clinton
- In 2017, US does not publicly and explicitly disavow One China policy
- In 2017, US does not withdraw from large trade org like WTO or NAFTA
- In 2017, US lifts at least half of existing sanctions on Russia
- In 2017, US will not get involved in any new major war with death toll of > 100 US soldiers
- In 2017, Ukraine will neither break into all-out war or get neatly resolved
- In 2017, a significant number of believers will not become convinced EMDrive doesn’t work
- In 2017, a significant number of skeptics will not become convinced EMDrive works
- In 2017, construction on Mexican border wall (beyond existing barriers) begins
- In 2017, no terrorist attack in any First World country will kill > 100 people
- In 2017, no terrorist attack in the USA will kill > 100 people
- In 2017, there will be no major civil war in Middle Eastern country not already experiencing a major civil war at the beginning of 2017
- Introduction to Bayes' Rule odds form
This is an arc that includes just enough content to teach about Bayes's Rule odds form.
- It's better to donate now rather than later.
- It's better to give $1000 to one person one time than to lend it out through microloans and then, as the money's repaid, keep relending it to other people indefinitely
A claim about microloans.
- It's easier to make scientific progress in countries with fewer regulations.
- It's good for GiveWell and Good Ventures to crowd out donors by their donations.
- Keith Ellison will be chosen as new DNC chair in 2017
- Kickstarter project is a better tool for fundraising a threshold amount of money to start an EA project than a donor charity
- Less Wrong
A community blog devoted to refining the art of human rationality.
- Less Wrong renaissance attempt will seem less (rather than more) successful by end of 2017
- Libya will remain a mess in 2017
- List
Meta tags for pages that are basically lists.
- Machine Intelligence Research Institute
Where to work if you're doing more formal or technical work on AI safety, of a kind not easily milked for publications.
- Marine Le Pen will not be elected President of France in 2017
- Market monetarism
Domain for school of macroeconomic thought.
- Math playpen
Playpen page for Math domain
- Meta tags which suppress a page from being featured
This parent collects all meta tags which will prevent a page from being listed in the featured secti…
- Mic-Ra-finance and the illusion of control
This post discusses the following claims:
* [claim([6th])]
* [claim([6tk])]
* [claim([6tl])]
- Microlending
The practice of giving microloans, which are small loans that are issued by individuals.
- Molecular basis of the potential of vitamin D to prevent cancer
Theoretical meta-analysis
Link: http://www.researchgate.net/publication/5812584_Molecular_basis_of_…
- More about Arbital
Lots more information about Arbital vision.
- More comments and claims should be driven by cruxes.
- Moving outside of the US will likely result in cheaper accommodations and living costs.
- Needs exercises
Add this tag to a page which doesn't have enough exercises.
- Needs summary
This page does not have a summary which provides an informative overview of the page's primary topic.
- Neutral genie metaphor
Definition. A neutral-genie metaphor is an attempt to illustrate a possible formal problem via an in…
- No agreement will be reached on "two-speed EU"
- No country currently in Euro or EU will announce new plan to leave in 2017
- No exchange of fire over "tiny stupid islands" in 2017
- No major earthquake (>10,000 deaths) in the world in 2017
- No major earthquake (>100 deaths) in US in 2017
- No major war in Asia (with >100 Chinese, Japanese, South Korean, and American deaths combined) over "tiny stupid islands" in 2017
- No serious impeachment proceedings are active against Trump in 2017
- Now I am become Life, the protector of worlds
In which Arbital is brought forth into the world.
- Odds: Technical explanation
Formal definitions, alternate representations, and uses of odds and odds ratios (like a 1 : 2 chance of drawing a red ball vs. green ball from a barrel).
- Oil will end 2017 higher than $50 a barrel
- Oil will end 2017 lower than $60 a barrel
- On the importance of Less Wrong, or another single conversational locus
In this post, Anna Salamon talks about how Less Wrong used to a locus of discussion, and that it is…
- Organizations and people
Comprehensive list of organizations and people relevant to navigating development of AGI.
- Our community should relocate to Iceland.
- Our community should relocate to Japan.
- Our community should relocate to a country other than the US
Claims with specific country proposals:
* [claim([6v8])]
* [claim([6v9])]
Relevant claims:
* [cla…
- Out of date
Meta tag used when the page has a lot of information that's obsolete
- Outside view
Taking the **outside view** (another name for reference class forecasting) means using an estimate b…
- Page's title should always be capitalized
Vote "agree" if you think Arbital should enforce the first letter of a page title to always be capit…
- Policy desiderata in the development of machine superintelligence
Link to the paper by Nick Bostrom, Allan Dafoe, and Carrick Flynn.
- Predictions For 2017
Scott Alexander made 105 predictions for 2017. Most of them are not personal and are listed below. …
- Project proposal: Complex numbers
Project proposal for complex numbers
- Quality meta tags
Meta tags which determine the page's quality.
- Questionnaire helper 1
Just a helper page to demonstrate Arbital's questionnaire features.
- Questionnaire helper 2
Just a helper page to demonstrate Arbital's questionnaire features.
- Questionnaire helper 3
Just a helper page to demonstrate Arbital's questionnaire features.
- SSC will remain active by end of 2017
- Science Survey
Nothing here yet.]
Automatically generated page for "Science Survey" group.
If you are the owner, …
- Serum 25-Hydroxyvitamin D Levels and the Prevalence of Peripheral Arterial Disease
Clinical and population study
Link: http://atvb.ahajournals.org/content/28/6/1179.full
From conclu…
- Serum 25-Hydroxyvitamin D and Risks of Colon and Rectal Cancer in Finnish Men
Case-control study.
Link: http://aje.oxfordjournals.org/content/173/5/499.full
From discussion:
>…
- Serum Vitamin D Concentration and Prostate Cancer Risk: A Nested Case – Control Study
Prospective analysis.
Link: http://jnci.oxfordjournals.org/content/100/11/796.full.pdf
From conclu…
- Shanghai index will not end 2017 down by more than 10%
- Situation in Israel will look more worse than better by end of 2017
- SlateStarCodex will have more than 15,000 Twitter followers by end of 2017
- Small donors are more likely to do other good if crowded out
- Some men just want to watch the world learn
In which Arbital gets paths.
- Some page with todos
Lots of things to do
- Stages
List of stages in AGI development and their properties.
- Style guidelines
Various stylistic conventions people should follow on Arbital
- Submitting a page to a domain
How and why to submit a page to a domain
- Syria’s civil war will not end in 2017
- Team Arbital
The people behind Arbital
- The UK will trigger Article 50 in 2017
- The Value of Coordination
>When you’re part of a community, the counterfactuals become more complex, and doing the most good b…
- The benefits of moving the community won't be realized in time before AGI.
- The community would lose a lot of its influence if it doesn't have a strong presence in key geographical locations (e.g. Bay Area).
- The current message of effective altruism heavily discourages creativity.
Alyssa Vance expands on this point in her [FB post](https://www.facebook.com/alyssamvance/posts/1021…
- The effect of vitamin D supplementation on skeletal, vascular, or cancer outcomes: a trial sequential meta-analysis
Trial sequential meta-analysis.
Link: http://www.thelancet.com/journals/landia/article/PIIS2213-858…
- The plan
Root page for the plan on how to approach and navigate through AGI development.
- The plan experiment
Root page for describing the reason and the process for planning how to approach and navigate through AGI development.
- The plan should be public
There is an open question of whether or not The plan experiment should be public. My current intuiti…
- There will be no Cast Lead style bombing/invasion of Gaza in 2017
- There will be no announcement of genetically engineered human baby or credible plan for such in 2017
- There will be no race riot killing > 5 people in 2017
- Theresa May will remain PM of Britain in 2017
- To make faster intellectual progress, it would help if the community was located more densely together.
- To math explanations and beyond!
In which Arbital doubles down on math explanations.
- To think thoughts fully uninfluenced by the surrounding society one has to be physically removed from it.
This claim is related to Our community should relocate to a country other than the US.
- Too long
Meta tag used to indicate that this page is too long by Arbital's standards.
- US unemployment to be higher at end of 2017 than beginning
- Unemployment and Automation
#WORK IN PROGRESS
I'm going to research and explore the topic of unemployment and automation. Quest…
- Useless variable decomposition
A variable decomposition can be true but useless if it is a poor guide to intervention due to automa…
- VAT playpen
Playpen page for VAT domain.
- Vitamin D
Vitamin D refers to a group of fat-soluble secosteroids responsible for enhancing intestinal absorpt…
- Vitamin D Deficiency and Risk for Cardiovascular Disease
Theoretical meta-analysis.
Link: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2726624/
From conclus…
- Vitamin D and Cardiometabolic Outcomes: A Systematic Review
Meta-analysis.
Link: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3211092/
From conclusions:
> "An…
- Vitamin D and calcium supplementation reduces cancer risk: results of a randomized trial
Randomized trial
Link: http://www.ncbi.nlm.nih.gov/pubmed/17556697%20
From conclusions:
> "Improv…
- Vitamin D helps prevent bone fracture
(Yes) One randomized trial shows some promise, but at extremely high doses of 100,000 IUs.
(No) A l…
- Vitamin D helps prevent breast cancer
No
--
Calcium and vitamin D supplementation did not reduce invasive breast cancer incidence in post…
- Vitamin D helps prevent cancer
There have been a lot of studies performed that show that vitamin D helps prevent cancer, but overal…
- Vitamin D helps prevent cardiovascular disease
This question is a bit vague. There are two ways to make it more specific.
1. Does vitamin D deffic…
- Vitamin D helps prevent colorectal cancer
No
--
Daily supplementation of calcium with vitamin D for seven years had no effect on the incidenc…
- Vitamin D helps prevent prostate cancer
Yes
---
Calcitriol showed significant antineoplastic activity in pre-clinical models of prostate ca…
- Vitamin D is good for you
We'll consider two categories of vitamin D supplementation: below and above the recommended levels.
…
- Vitamin D is related to blood pressure and other cardiovascular risk factors in middle-aged men
Correlational study
Link: http://www.ncbi.nlm.nih.gov/pubmed/8541004
- Vitamin, Mineral, and Multivitamin Supplements for the Primary Prevention of Cardiovascular Disease and Cancer: A Systematic Evidence Review for the US Preventive Services Task Force
Meta-study
Link: http://www.ncbi.nlm.nih.gov/pubmed/24308073
From results:
> "Vitamin D and/or ca…
- Welcome to Arbital
Front page explaining what Arbital is all about.
- What is the probability that impeachment proceedings will be commenced against President Donald Trump during his first term?
More on impeachment in United States: https://en.wikipedia.org/wiki/Impeachment_in_the_United_States
- What should superintelligence be programmed to do?
This is one of the key open questions in The plan experiment.
Coherent extrapolated volition (align…
- When I donate to a charity, I am concerned whether or not the charity will raise enough money to make my donation worthwhile.
- Why waiting to donate harms charities
A blog post explaining the potential reasons why someone would choose to wait to donate and how that leads to suboptimal outcomes for the charity.
- Will Narrow AI Seriously Affect Long-Term Employment?
###Definitions:
**Narrow AI** - artificial intelligence capable of doing a narrow task at a level …
- With some fixed amount of money to start, a microloan charity could make loans indefinitely
A claim about microloans.
- Working out, weightlifting, and nutrition
Just a collection of links / resources I'll be using to seed some content.
# Working out
### How t…
> The notification showed me my post rather than the comment.
Fixed
1. Not yet. You can just do footnotes like "\[1\]" or make them links, like [\[1\]](https://arbital.…
> But if we can keep our eyes out for a low-effort way to solve the problem, the return still feels …
> But we generally give priority to deeper generalizations continuing, i.e., generalizations that ar…
> Each leader takes small steps to avoid trading off others' goals too aggressively.
This seems har…
> I feel it's basically good to be straightforward, and also good to be in motion rather than waitin…
> On the other hand, for a startup-style outreach-focused project, substantial value comes from the …
Agree that it doesn't make sense for this to be a probability bar.
Agreed. Blogging will be one of the major areas we focus on after the announcement. (He keeps saying…
Agreed. Want to do that?
Ah, To think thoughts fully uninfluenced by the surrounding society one has to be physically removed…
Ah, interesting. Do you mean: "Once you found a working business / movement model, it's time to redu…
Ah... May be just put them at the bottom of the page. You an also wrap it in the %%coment%% syntax, …
All good points. I've updated my vote.
Anna might have different definitions, but here are mine:
> correct credit-tracking
Basically, the…
Apparently it's "which conscious states feel good, which ones feel bad?"
Awesome! Yeah, I'd love to discuss this more when we'll start implementing this feature (later this …
Child, because "Nick Bostrom is a person" and "Nick is a part of the 'people' object" and without "N…
Consider creating a blog page called "Nate's Blog" or something, and making this page a child of it.
Definitely have a strong feeling of wanting to read a summary instead of the entire article.
Do you mean "responses" or "votes"? Here is the claim for votes: Arbital should hide probability/app…
Even if the platform had pros and cons, you'd still need to decide for any given claim whether some …
First editor comment!
Fixed, thanks!
Fixed, thanks.
Fixed.
For now we'll just do Arbital on invite-only basis. This feature is still pretty far off in the futu…
From [this paper's](http://marginalrevolution.com/marginalrevolution/2015/12/the-rise-and-decline-of…
Funny enough we had something like that in an older version. We'll definitely bring it back. One way…
Good point. I think Kickstarter campaigns are gamified in all sorts of ways. For example, having a l…
Good point. We'll probably have different UI for them at some point.
Good question! I'll think about this, but your proposed solution sounds like a good start.
Good question. We don't have table or HTML support right now, but I"ll bump up the priority on that.…
Had to re-read this twice. Not sure if I fully got it.
Hah, yeah, agreed. I'm going to add more options / info to the children list, allow for sorting opti…
Hmm, can't reproduce. You were on the /edit/ page?
Hmm, my feeling is that claims should usually be refactored to be more useful for more people. I gue…
How do you think it plays out in one million years?
Huh... Not sure I understand this. I have BS in CS, but don't remember running across this. Would lo…
I basically agree with everything here.
I definitely think something like this should exist and will be helpful, but I think Arbital should …
I disagree mostly on priors, since it's quite unlikely that we discovered, understood, and pinpointe…
I feel like Paul Crowley's version is basically the same as this one.
And yes, I agree that there a…
I have an intuition that says that if you run any sufficiently large computation (even if it's as si…
I second the "I also moderately believe that most EAs will not use the extra time very well" claim.
I think "universally" is too strong a word, but I agree that most of the time it's the right thing t…
I think if one was to attempt it, it would be important to get the messaging right. If you just did …
I think it's inevitable that we'll need to build our own editor. I'm not at all sure what that will …
I think that would definitely capture a big part of it. There are also fun facts that are interestin…
I think there is a version of this claim that's basically trivially true and a version that's intere…
I updated my vote in response to Rob Bensinger's comment.
Regarding that claim, there is a related …
I was slightly confused for the next two paragraphs, because I had a silly thought that 10,001 uses …
I'm curious if the inverse has any particular use in this field.
I'm not sure how you define "smooth." Continuous? Differentiable?
I'm noticing a trend of pages with titles that start with a lower case letter. Is that on purpose? S…
I'm very confused why you need two links to the same page (and one of them is blue).
Ideally we shouldn't have pages like this. It means that the hierarchy feature failed. Is this just …
If you look on Wikipedia's page about [up arrow](https://en.wikipedia.org/wiki/Knuth%27s_up-arrow_no…
Inline comment.
Is [0, inf) same as R+?
It's a totally made up, arbitrary syntax that makes no sense and NEEDS TO GO.
It's not clear to me what point you are making here.
It's super standard [explorer type look](https://docs.oracle.com/cd/E21043_01/doc.1111/e10624/img/in…
Likes work as substitutes. More granular likes and more variety of expressions (ala FB reactions) wo…
Looks like at some point one of our scripts did something funky to this page (probably replaced `Ale…
Looks like there is a typo with the fraction. sense 1...n . 1?
May be there are two or more target tracks. One for small donations(who donate \$1k or less) and oth…
Most technical version goes onto the primary page (this one). Easier versions get their own lenses. …
My main point of disagreement is that it's sometimes hard to say if some evidence is "for" or "again…
My take on it: let's say you tell the other person you are running 10 minutes late. If you end up co…
Narrowness is a virtue, especially in mathematics. The tighter and more precise you can make your st…
New comment2.
New comment3.
Nice! I've never seen an example like this before, and it's actually kind of surprising. It seems to…
No. My intention there wasn't to ask for help, and in fact, I'd prefer to be clear that we don't nee…
Not yet.
Oh, not at all. Probably a bug; I'm going to look into it right now.
Ok, Eliezer, you've addressed my point directly with sapience_0 / sapience_1 example. That makes sen…
Okay now I'm also confused. (Eric Rogstad)
Why don't we just say its codomain is {1}?
Okay. I hope to do that with meta tags in the future, e.g. "stub".
One can imagine an agent that is smart about finding and training itself on new features. You seed i…
Paul, you can start by writing an objection as a comment, if it's a few paragraphs long. You can wri…
People can already delete their pages on Arbital. You can bring it back by reverting to a previous e…
Pivotal? Game-changing? Terminal?
Please delete this if you are no longer using it. If you are, let me know how.
Reason for my disagreement with A typical donor with a $5,000 charity budget, on the margin, has inc…
Right on both counts. I've changed the wording.
Seems pretty odd for this to rely on Bayes Rule. Is that just a temporary thing?
Sharing is caring!!
Should be fixed. Try now.
Smallest?
So then should the title of the page be "Bayesian prior"?
Sometimes ambiguous claims can be good too. Just to get a quick sense of where people are at. And fo…
Sounds good!
The invite text is a bit unclear. You don't need to do anything; you already have the …
Sounds good, I'd love to see a mockup. Eliezer Yudkowsky, might have ideas about this.
Sounds like we could capture most of those wins via, for example, Our community should relocate to a…
Talk to me: alexei@arbital.com
Test comment.
Test
Test.
Testing comments in Firefox.
Thanks for bringing this up. The \$ issue is relatively new, since we've been changing the editor a …
Thanks for bringing this up. I fixed the core part of the issue: you can now use backticks to escape…
Thanks for the comment! Don't hesitate to edit the page.
You can also edit your comment by hoveri…
Thanks for the critique, Ted. We are currently figuring out the life-cycle of a claim, and will find…
Thanks for the feedback. :)
Yeah, there is a lot left to do on the comment side. That's going to b…
Thanks, changed!
Thanks, changed. Now you have the power to edit the pages yourself. ;)
The formula uses "x"s, but should it use "f"s instead?
The what, the huh?
They are just not related. No enforcement of any kind is made.
This can be formatted better. Also I think there is a typo with `else 0`?
This is awkward phrasing.
This is interesting! Is there a name for this concept / area of thought?
This is not a very good summary, since it relies on the reader understanding what a "complete number…
This is probably explained elsewhere, but what's AIXI-**tl**?
This makes me think that perhaps more, smaller milestones would be good.
This paragraph is hard to understand. May be rewrite it in a more concrete way, i.e. using "is 91 pr…
This should probably be re-titled to "Needs splitting by mastery" or something. "Needs splitting by …
This sounds interesting! Would love to read more about this.
To quote Anna: the "aspiring rationality" community; the "effective altruist" project; efforts to cr…
Typo: "andi nefficient"
We removed that button from the quick menu because it had too many buttons. Now you have to create a…
What **is** life??
What's $n$ exactly?
What's valence research?
When/if you figure it out, feel free to change.
Would be nice to be able to see explicitly why this is so. Not everyone can do powers of two in thei…
Yeah, I think More comments and claims should be driven by cruxes..
Yeah, I'm still very confused on this subject. At some point I'll get back and add more data / point…
Yeah, good points! We are still figuring out the norms for the discussion, and how it will ultimatel…
Yeah, that's a bug.
Yes, hard delete will be a thing too.
Yes! That's going to be the major priority in the upcoming months.
Yes, but the difference between reals and positive reals isn't that big. However, I might be confuse…
Yes, but there is not a lot of content, so I'm keeping it as one for now.
Yes, yes, yes. I've recently realized these things too. Very much agree.
Yes.
You should make a claim, because I think I disagree. For example, there is a threshold at which they…
Yup! Eliezer Yudkowsky
Yup, we'll add that as an option at some point.
"path" is the word we usually use.
Andrew Critch, these are all the Arbital-specific markdown addons we have.
Anton Geraschenko, Danny An, Rafael Cosman testing the fix for the incorrectly formatted user popove…
Kevin Clancy, did you intend to make this a blog page (owned by you) as opposed to a wiki page (owne…
Eliezer Yudkowsky, Andrew Critch, please read, since you'll be creating most of the content. One of …
Robert Lecnik Calcitriol in the Treatment of Prostate Cancer
Robert Lecnik mention
Eric Rogstad's stark disagreement made me realize that there are two ways to interpret this question…
Eric Rogstad, I think the claims you created for this page are too specific / literal. Which means t…
Stephanie Zolayvar, looks like several people don't get this. Can you qualify / rewrite?
https://arbital.com/p/6mt/ => If we can’t lie to others, we will lie to ourselves
in X, **such that**...
test
test
test
question
- What does a sample question look like?
Just an example of an Arbital question.
- Where is a playpen question?
Sandbox / playpen question page.
wiki
wiki
After reading the Doc(tm), I think there is still design space to explore. For most readers, and ma…
Again I think I erred in including "reputation system" in the claim. I was trying to draw a distinc…
Good catch about reputation being critical. Perhaps I should remove that, since I'd like to see rep…
Great analysis of problems with TruthSift. Perhaps we should start a list of irregular and natural…
I imagine you could propose evidence as "for" or "against", and then the discussion steps down a lev…
I see reputation systems as being necessary, but not sufficient. Without argument structuring featu…
I would rather a claim is always in a clarification period. If a claim can't be modified or varied,…
If claims are primitives, then all the interesting conversations will be at a parent level, which wi…
Let me experiment with using a Page for this purpose, and see what seems like it's missing.
I think…
Nice. Did Braintropes see the tabsplosions as a good thing or a bad thing?
So, how do we measure value in elevating accuracy or truth?
Sounds right, but this "page" you speak of is new to me. I assume it's the base structure of the ma…
Thanks for picking apart my claim, folks! Rather than modify this claim, I think I'll work on a Pos…
Thanks for those links. My view is partly inspired by the first post, and the second is new to me. …
This is great. I think my next core argument needs to be for why argument structuring is more than …
wiki
- Correct credit-tracking is very important if we want our community to generate new good ideas.
Correct credit-tracking is very important if we want our community to generate new good ideas.
wiki
- Evolution Strategies and Reinforcement Learning
Evolution strategies as a simplified implementation of reinforcement learning
wiki
- An early stage prioritisation model
How do you choose which projects to work on, early on in life?
- An early stage project generation model
How do you figure out which projects to even consider, when you're getting started?
- Early-life planning
How do you figure out what to do at the beginning of your life?
- Open questions in life-planning
What questions do the two models above not solve?
wiki
- A typical donor with a $5,000 charity budget, on the margin, has increasing returns to scale
- Claim explainer: donor lotteries and returns to scale
Sometimes, new technical developments in the discourse around effective altruism can be difficult to…
- GiveWell and the problem of partial funding
A foundation that plans to move around ten billion dollars and is relying on advice from GiveWell isn’t enough to get the top charities fully funded. That’s weird and surprising.
- Good Ventures has increasing returns to scale
- Improve comments by tagging claims
Comment sections are more important for discourse than I thought. They can be improved by explicitly tagging an article's main claims as anchors for discussion.
- Location on the comments-links continuum is an important aspect of discourse design.
wiki
- Metric
A metric is a function that defines a distance between elements in a set and follows some basic rules.
- Existential risk
one where an adverse outcome would either **annihilate Earth-originating intelligent life** or perma…
wiki
- Primer on Infinite Series
What does it mean to add things together forever?
wiki
- Curtis's Blog
I'm testing the blogging functionality.
- Mutual information
http://colah.github.io/posts/2015-09-Visual-Information/
wiki
- Concrete groups (Draft)
Instead of thinking of a group as a set with operations satisfying axoims, we develop groups as symmetry groups of various objects
- Groups as symmetires
A group is an abstraction of a collection of symmetries of an object. Examples of groups include th…
- Power set
The power set is the collection of all subsets of a set
wiki
- Effectiveness of scaling and root planing for gingivitus/periodontal disease
Scaling and root planing (SRP) is a dental procedure that involves scraping stuff off of your teeth …
- Harem
A group of women loving their superman.
- Mathematical induction
Proving a statement about all positive integers by knocking them down like dominoes.
- A beginner's guide to explaining things
Good explanations can be very different, but most of them have a few things in common.
- An algorithm for creating good explanations
Excellent explanations can break the rules, but it helps to know those rules in the first place. Following these steps will help you create explanations that are solid and that avoid common mistakes.
- Audience-centric Explanations
A good explanation isn't one that says all the things, but one that allows all the things to be HEARD.
- Confirming understanding
Attempts to explain a concept should be falsifiable
- Core concepts for creating good explanations
A moderately deep dive into the concepts that underlie understanding and how to create it.
- Examples of good explanations
A representative variety of good explanations, along with analysis of what each explanation does well (or not).
- How understanding is made
How analogy and concept maps help us acquire and retain knowledge
- Knowing your audience
A brief introduction to the concepts of assessment and pedagogical content knowledge
- Lt. Gilbert S. Daniels and the myth of averages
There is no such thing as an average pilot
- Meeting your audience
Framing explanations, overcoming inferential distance, and effective differentiation
- Tools and tactics for improving explanations
A list of some of the most useful and generally applicable techniques for creating understanding.
wiki
- 0.999...=1
No, it's not "infinitesimally far" from 1 or anything like that. 0.999... and 1 are literally the same number.
- Church encoding
How can you represent things like numbers as lambda expressions?
- Equivalence relation
A relation that allows you to partition a set into equivalence classes.
- Lambda calculus
A minimal, inefficient, and hard-to-read, but still interesting and useful, programming language.
- Math style guidelines
Stylistic conventions specific to pages about math.
- Ordered ring
A ring with a total ordering compatible with its ring structure.
- Rice's Theorem: Intro (Math 1)
You can't write a program that looks at another programs source code, and tells you whether it computes the Fibonacci sequence.
- Subgroup
A group that lives inside a bigger group.
- The square root of 2 is irrational
The number whose square is 2 can't be written is a quotient of natural numbers
- Transitive relation
If a is related to b and b is related to c, then a is related to c.
- Well-ordered set
An ordered set with an order that always has a "next element".
- Complex number
A complex number is a number of the form $z = a + b\textrm{i}$, where $\textrm{i}$ is the imaginary …
wiki
- 'Beneficial'
Really actually good. A metasyntactic variable to mean "favoring whatever the speaker wants ideally to accomplish", although different speakers have different morals and metaethics.
- 'Concept'
In the context of Artificial Intelligence, a 'concept' is a category, something that identifies thingies as being inside or outside the concept.
- 'Detrimental'
The opposite of beneficial.
- 'Rationality' of voting in elections
"A single vote is very unlikely to swing the election, so your vote is unlikely to have an effect" versus "Many people similar to you are making a similar decision about whether to vote."
- 99LDT x 1CDT oneshot PD tournament as arguable counterexample to LDT doing better than CDT
Arguendo, if 99 LDT agents and 1 CDT agent are facing off in a one-shot Prisoner's Dilemma tournament, the CDT agent does better on a problem that CDT considers 'fair'.
- A quick econ FAQ for AI/ML folks concerned about technological unemployment
Yudkowsky's attempted description of standard economic concepts that he thinks are vital for talking about technological unemployment and related issues.
- A reply to Francois Chollet on intelligence explosion
A quick run-through of what I'd consider the standard replies to the arguments in Keras inventor Francois Chollet's essay "The impossibility of intelligence explosion".
- AI alignment
The great civilizational problem of creating artificially intelligent computer systems such that running them is a good idea.
- AI alignment open problem
Tag for open problems under AI alignment.
- AI arms races
AI arms races are bad
- AI safety mindset
Asking how AI designs could go wrong, instead of imagining them going right.
- AIXI
How to build an (evil) superintelligent AI using unlimited computing power and one page of Python code.
- AIXI-tl
A time-bounded version of the ideal agent AIXI that uses an impossibly large finite computer instead of a hypercomputer.
- Ability to read algebra
Do you have sufficient mathematical ability that you can read a sentence that uses some algebra or invokes a mathematical idea, without slowing down too much?
- Ability to read calculus
Can you take integral signs and differentiations in stride?
- Ability to read logic
Can you read sentences symbolically stating "For all x: exists y: phi(x, y) or not theta(y)" without slowing down too much?
- Abortable plans
Plans that can be undone, or switched to having low further impact. If the AI builds abortable nanomachines, they'll have a quiet self-destruct option that includes any replicated nanomachines.
- Absent-Minded Driver dilemma
A road contains two identical intersections. An absent-minded driver wants to turn right at the second intersection. "With what probability should the driver turn right?" argue decision theorists.
- Actual effectiveness
If you want the AI's so-called 'utility function' to actually be steering the AI, you need to think about how it meshes up with beliefs, or what gets output to actions.
- Ad-hoc hack (alignment theory)
A "hack" is when you alter the behavior of your AI in a way that defies, or doesn't correspond to, a principled approach for that problem.
- Advanced agent properties
How smart does a machine intelligence need to be, for its niceness to become an issue? "Advanced" is a broad term to cover cognitive abilities such that we'd need to start considering AI alignment.
- Advanced nonagent
Hypothetically, cognitively powerful programs that don't follow the loop of "observe, learn, model the consequences, act, observe results" that a standard "agent" would.
- Advanced safety
An agent is *really* safe when it has the capacity to do anything, but chooses to do what the programmer wants.
- Algorithmic complexity
When you compress the information, what you are left with determines the complexity.
- Aligning an AGI adds significant development time
Aligning an advanced AI foreseeably involves extra code and extra testing and not being able to do everything the fastest way, so it takes longer.
- Almost all real-world domains are rich
Anything you're trying to accomplish in the real world can potentially be accomplished in a *lot* of different ways.
- An Introduction to Logical Decision Theory for Everyone Else
So like what the heck is 'logical decision theory' in terms a normal person can understand?
- Answer to sparking widgets problem
Odds of 1 : 3, probability of 1/4.
- Arbital biographies
As a very strong default (presently an absolute rule), Joe Smith's page only says nice things about Joe. Even if a negative fact is true, it doesn't go on Joe's page.
- Arbital playpen
Want to test a feature? Feel free to edit this page! asdfasfdasfda
- Arbital practices
Guidelines and rules for interacting on Arbital.
- Arbital: Solving online explanations
An explanation of Arbital's mid-term goals
- Arithmetical hierarchy
The arithmetical hierarchy is a way of classifying logical statements by the number of clauses saying "for every object" and "there exists an object".
- Arithmetical hierarchy: If you don't read logic
The arithmetical hierarchy is a way of stratifying statements by how many "for every number" and "th…
- Artificial General Intelligence
An AI which has the same kind of "significantly more general" intelligence that humans have compared to chimpanzees; it can learn new domains, like we can.
- Attainable optimum
The 'attainable optimum' of an agent's preferences is the best that agent can actually do given its finite intelligence and resources (as opposed to the global maximum of those preferences).
- Autonomous AGI
The hardest possible class of Friendly AI to build, with the least moral hazard; an AI intended to neither require nor accept further direction.
- Averting instrumental pressures
Almost-any utility function for an AI, whether the target is diamonds or paperclips or eudaimonia, implies subgoals like rapidly self-improving and refusing to shut down. Can we make that not happen?
- Averting the convergent instrumental strategy of self-improvement
We probably want the first AGI to *not* improve as fast as possible, but improving as fast as possible is a convergent strategy for accomplishing most things.
- Bayes' rule
Bayes' rule is the core theorem of probability theory saying how to revise our beliefs when we make a new observation.
- Bayes' rule examples
Interesting problems solvable by Bayes' rule
- Bayes' rule: Functional form
Bayes' rule for to continuous variables.
- Bayes' rule: Guide
The Arbital guide to Bayes' rule
- Bayes' rule: Log-odds form
A simple transformation of Bayes' rule reveals tools for measuring degree of belief, and strength of evidence.
- Bayes' rule: Odds form
The simplest and most easily understandable form of Bayes' rule uses relative odds.
- Bayes' rule: Odds form (Intro, Math 1)
Introduction to the odds form of Bayes' rule
- Bayes' rule: Odds form (Intro, Probability)
Intro to Bayes' rule, odds form, for people already familiar with probability.
- Bayes' rule: Proportional form
The fastest way to say something both convincing and true about belief-updating.
- Bayes' rule: Vector form
For when you want to apply Bayes' rule to lots of evidence and lots of variables, all in one go. (This is more or less how spam filters work.)
- Bayesian reasoning
A probability-theory-based view of the world; a coherent way of changing probabilistic beliefs based on evidence.
- Bayesian update
Bayesian updating: the ideal way to change probabilistic beliefs based on evidence.
- Bayesian view of scientific virtues
Why is it that science relies on bold, precise, and falsifiable predictions? Because of Bayes' rule, of course.
- Behaviorist genie
An advanced agent that's forbidden to model minds in too much detail.
- Belief revision as probability elimination
Update your beliefs by throwing away large chunks of probability mass.
- Big-picture strategic awareness
We start encountering new AI alignment issues at the point where a machine intelligence recognizes the existence of a real world, the existence of programmers, and how these relate to its goals.
- Bounded agent
An agent that operates in the real world, using realistic amounts of computing power, that is uncertain of its environment, etcetera.
- Boxed AI
Idea: what if we limit how AI can interact with the world. That'll make it safe, right??
- Bulverism
Bulverism is when you explain what goes so horribly wrong in people's minds when they believe X, before you've actually explained why X is wrong. Forbidden on Arbital.
- Calories-In-Calories-Out
CICO is a proposed conceptual decomposition of the causes of changes in human body mass, particularl…
- Cartesian agent
Agents separated from their environments by impermeable barriers through which only sensory information can enter and motor output can exit.
- Cartesian agent-environment boundary
If your agent is separated from the environment by an absolute border that can only be crossed by sensory information and motor outputs, it might just be a Cartesian agent.
- Causal decision theories
On CDT, to choose rationally, you should imagine the world where your physical act changes, then imagine running that world forward in time. (Therefore, it's irrational to vote in elections.)
- Central examples
The "central examples" for a subject are examples that are referred to over and over again in the co…
- Central examples
List of central examples in Value Alignment Theory domain.
- Cognitive domain
An allegedly compact unit of knowledge, such that ideas inside the unit interact mainly with each other and less with ideas in other domains.
- Cognitive steganography
Disaligned AIs that are modeling human psychology and trying to deceive their programmers will want to hide their internal thought processes from their programmers.
- Cognitive uncontainability
'Cognitive uncontainability' is when we can't hold all of an agent's possibilities inside our own minds.
- Coherence theorems
A 'coherence theorem' shows that something bad happens to an agent if its decisions can't be viewed as 'coherent' in some sense. E.g., an inconsistent preference ordering leads to going in circles.
- Coherent decisions imply consistent utilities
Why do we all use the 'expected utility' formalism? Because any behavior that can't be viewed from that perspective, must be qualitatively self-defeating (in various mathy ways).
- Coherent extrapolated volition (alignment target)
A proposed direction for an extremely well-aligned autonomous superintelligence - do what humans would want, if we knew what the AI knew, thought that fast, and understood ourselves.
- Complexity of value
There's no simple way to describe the goals we want Artificial Intelligences to want.
- Conceivability
A hypothetical scenario is 'conceivable' or 'imaginable' when it is not *immediately* incoherent, al…
- Conditional probability
The notation for writing "The probability that someone has green eyes, if we know that they have red hair."
- Consequentialist cognition
The cognitive ability to foresee the consequences of actions, prefer some outcomes to others, and output actions leading to the preferred outcomes.
- Consequentialist preferences are reflectively stable by default
Gandhi wouldn't take a pill that made him want to kill people, because he knows in that case more people will be murdered. A paperclip maximizer doesn't want to stop maximizing paperclips.
- Conservative concept boundary
Given N example burritos, draw a boundary around what is a 'burrito' that is relatively simple and allows as few positive instances as possible. Helps make sure the next thing generated is a burrito.
- Context disaster
Some possible designs cause your AI to behave nicely while developing, and behave a lot less nicely when it's smarter.
- Convergent instrumental strategies
Paperclip maximizers can make more paperclips by improving their cognitive abilities or controlling more resources. What other strategies would almost-any AI try to use?
- Convergent strategies of self-modification
The strategies we'd expect to be employed by an AI that understands the relevance of its code and hardware to achieving its goals, which therefore has subgoals about its code and hardware.
- Coordinative AI development hypothetical
What would safe AI development look like if we didn't have to worry about anything else?
- Corporations vs. superintelligences
Corporations have relatively few of the advanced-agent properties that would allow one mistake in aligning a corporation to immediately kill all humans and turn the future light cone into paperclips.
- Correlated competency
When an AI achieving sufficiently high goodness on behavior A means we should strongly expect high goodness on behavior B.
- Correlated coverage
In which parts of AI alignment can we hope that getting many things right, will mean the AI gets everything right?
- Cosmic endowment
The 'cosmic endowment' consists of all the stars that could be reached from probes originating on Earth; the sum of all matter and energy potentially available to be transformed into life and fun.
- Cosmopolitan value
Intuitively: Value as seen from a broad, embracing standpoint that is aware of how other entities may not always be like us or easily understandable to us, yet still worthwhile.
- Death in Damascus
Death tells you that It is coming for you tomorrow. You can stay in Damascus or flee to Aleppo. Whichever decision you actually make is the wrong one. This gives some decision theories trouble.
- Decision theory
The mathematical study of ideal decisionmaking
- Deep Blue
The chess-playing program, built by IBM, that first won the world chess championship from Garry Kasparov in 1996.
- Definition
Meta tag used to mark pages that strictly define a particular term or phrase.
- Descriptive versus normative propositions
A normative proposition talks about what should be; a descriptive proposition talks about what is.
- Development phase unpredictable
Several proposed problems in advanced safety are alleged to be difficult because they depend on some…
- Diamond maximizer
How would you build an agent that made as much diamond material as possible, given vast computing power but an otherwise rich and complicated environment?
- Difficulty of AI alignment
How hard is it exactly to point an Artificial General Intelligence in an intuitively okay direction?
- Directing, vs. limiting, vs. opposing
Getting the AI to compute the right action in a domain; versus getting the AI to not compute at all in an unsafe domain; versus trying to prevent the AI from acting successfully. (Prefer 1 & 2.)
- Diseasitis
20% of patients have Diseasitis. 90% of sick patients and 30% of healthy patients turn a tongue depressor black. You turn a tongue depressor black. What's the chance you have Diseasitis?
- Distances between cognitive domains
Often in AI alignment we want to ask, "How close is 'being able to do X' to 'being able to do Y'?"
- Distant superintelligences can coerce the most probable environment of your AI
Distant superintelligences may be able to hack your local AI, if your AI's preference framework depends on its most probable environment.
- Distinguish which advanced-agent properties lead to the foreseeable difficulty
Say what kind of AI, or threshold level of intelligence, or key type of advancement, first produces the difficulty or challenge you're talking about.
- Do-What-I-Mean hierarchy
Successive levels of "Do What I Mean" or AGIs that understand their users increasingly well
- Don't try to solve the entire alignment problem
New to AI alignment theory? Want to work in this area? Already been working in it for years? Don't try to solve the entire alignment problem with your next good idea!
- Edge instantiation
When you ask the AI to make people happy, and it tiles the universe with the smallest objects that can be happy.
- Effability principle
You are safer the more you understand the inner structure of how your AI thinks; the better you can describe the relation of smaller pieces of the AI's thought process.
- Emphemeral premises
When somebody says X, don't just say, "Oh, not-X because Y" and then forget about Y a day later. Y is now an important load-bearing assumption in your worldview. Write Y down somewhere.
- Empirical probabilities are not exactly 0 or 1
"Cromwell's Rule" says that probabilities of exactly 0 or 1 should never be applied to empirical propositions - there's always some probability, however tiny, of being mistaken.
- Environmental goals
The problem of having an AI want outcomes that are out in the world, not just want direct sense events.
- Epistemic and instrumental efficiency
An efficient agent never makes a mistake you can predict. You can never successfully predict a directional bias in its estimates.
- Epistemic exclusion
How would you build an AI that, no matter what else it learned about the world, never knew or wanted to know what was inside your basement?
- Epistemology
What is truth?
- Evidential decision theories
Theories which hold that the principle of rational choice is "Choose the act that would be the best news, if somebody told you that you'd chosen that act."
- Executable philosophy
Philosophical discourse aimed at producing a trustworthy answer or meta-answer, in limited time, which can used in constructing an Artificial Intelligence.
- Expected utility
Scoring actions based on the average score of their probable consequences.
- Expected utility agent
If you're not some kind of expected utility agent, you're going in circles.
- Expected utility formalism
Expected utility is the central idea in the quantitative implementation of consequentialism
- Explicit Bayes as a counter for 'worrying'
Explicitly walking through Bayes's Rule can summarize your knowledge and thereby stop you from bouncing around pieces of it.
- Extraordinary claims
What makes something an 'extraordinary claim' that requires extraordinary evidence?
- Extraordinary claims require extraordinary evidence
The people who adamantly claim they were abducted by aliens do provide some evidence for aliens. They just don't provide quantitatively enough evidence.
- Extrapolated volition (normative moral theory)
If someone asks you for orange juice, and you know that the refrigerator contains no orange juice, should you bring them lemonade?
- Fair problem class
A problem is 'fair' (according to logical decision theory) when only the results matter and not how we get there.
- Faithful simulation
How would you identify, to a Task AGI (aka Genie), the problem of scanning a human brain, and then running a sufficiently accurate simulation of it for the simulation to not be crazy or psychotic?
- Fallacies
To call something a fallacy is to assert that you think people shouldn't think like that.
- Finishing your Bayesian path on Arbital
The page that comes at the end of reading the Arbital Guide to Bayes' rule
- Flag the load-bearing premises
If somebody says, "This AI safety plan is going to fail, because X" and you reply, "Oh, that's fine because of Y and Z", then you'd better clearly flag Y and Z as "load-bearing" parts of your plan.
- Friendly AI
Old terminology for an AI whose preferences have been successfully aligned with idealized human values.
- General intelligence
Compared to chimpanzees, humans seem to be able to learn a much wider variety of domains. We have 'significantly more generally applicable' cognitive abilities, aka 'more general intelligence'.
- Generalized principle of cognitive alignment
When we're asking how we want the AI to think about an alignment problem, one source of inspiration is trying to have the AI mirror our own thoughts about that problem.
- Glossary (Value Alignment Theory)
Words that have a special meaning in the context of creating nice AIs.
- Goal-concept identification
Figuring out how to say "strawberry" to an AI that you want to bring you strawberries (and not fake plastic strawberries, either).
- Goodhart's Curse
The Optimizer's Curse meets Goodhart's Law. For example, if our values are V, and an AI's utility function U is a proxy for V, optimizing for high U seeks out 'errors'--that is, high values of U - V.
- Goodness estimate biaser
Some of the main problems in AI alignment can be seen as scenarios where actual goodness is likely to be systematically lower than a broken way of estimating goodness.
- Gotcha button
A conversational point which, when pressed, causes the other person to shout "Gotcha!" and leap on what they think is a weakness allowing them to dismiss the conversation.
- Guarded definition
A guarded definition is one where at least one position suspects there will be pressure to stretch a…
- Guide to Logical Decision Theory
The entry point for learning about logical decision theory.
- Happiness maximizer
It is sometimes proposed that we build an AI intended to maximize human happiness. (One early propo…
- Hard problem of corrigibility
Can you build an agent that reasons as if it knows itself to be incomplete and sympathizes with your wanting to rebuild or correct it?
- Harmless supernova fallacy
False dichotomies and continuum fallacies which can be used to argue that anything, including a supernova, must be harmless.
- High-speed intro to Bayes's rule
A high-speed introduction to Bayes's Rule on one page, for the impatient and mathematically adept.
- How to author on Arbital!
Want to contribute pages to Arbital? Here's our current version of the ad-hoc guide to being an author!
- How to build your own Lumenator
Treating Seasonal Affective Disorder using MOAR LIGHT can sometimes solve what dinky little lightboxes can't.
- Humans doing Bayes
The human use of Bayesian reasoning in everyday life
- Humean degree of freedom
A concept includes 'Humean degrees of freedom' when the intuitive borders of the human version of that concept depend on our values, making that concept less natural for AIs to learn.
- Hypercomputer
Some formalisms demand computers larger than the limit of all finite computers
- Ideal target
The 'ideal target' of a meta-utility function is the value the ground-level utility function would take on if the agent updated on all possible evidence; the 'true' utilities under moral uncertainty.
- Identifying ambiguous inductions
What do a "red strawberry", a "red apple", and a "red cherry" have in common that a "yellow carrot" doesn't? Are they "red fruits" or "red objects"?
- Identifying causal goal concepts from sensory data
If the intended goal is "cure cancer" and you show the AI healthy patients, it sees, say, a pattern of pixels on a webcam. How do you get to a goal concept *about* the real patients?
- Ideological Turing test
Can you explain the opposing position well enough that people can't tell whether you or a real advocate of that position created the explanation?
- Ignorance prior
Key equations for quantitative Bayesian problems, describing exactly the right shape for what we believed before observation.
- Imitation-based agent
An AI meant to imitate the behavior of a reference human as closely as possible.
- Immediate goods
One of the potential views on 'value' in the value alignment problem is that what we should want fro…
- Inductive prior
Some states of pre-observation belief can learn quickly; others never learn anything. An "inductive prior" is of the former type.
- Infrahuman, par-human, superhuman, efficient, optimal
A categorization of AI ability levels relative to human, with some gotchas in the ordering. E.g., in simple domains where humans can play optimally, optimal play is not superhuman.
- Instrumental
What is "instrumental" in the context of Value Alignment Theory?
- Instrumental convergence
Some strategies can help achieve most possible simple goals. E.g., acquiring more computing power or more material resources. By default, unless averted, we can expect advanced AIs to do that.
- Instrumental goals are almost-equally as tractable as terminal goals
Getting the milk from the refrigerator because you want to drink it, is not vastly harder than getting the milk from the refrigerator because you inherently desire it.
- Instrumental pressure
A consequentialist agent will want to bring about certain instrumental events that will help to fulfill its goals.
- Intelligence explosion
What happens if a self-improving AI gets to the point where each amount x of self-improvement triggers >x further self-improvement, and it stays that way for a while.
- Intended goal
Definition. An "intended goal" refers to the intuitive intention in the mind of a human programmer …
- Intension vs. extension
"Red is a light with a wavelength of 700 nm" vs. "Look at this red apple, red car, and red cup."
- Interest in mathematical foundations in Bayesianism
"Want" this requisite if you prefer to see extra information about the mathematical foundations in Bayesianism.
- Interruptibility
A subproblem of corrigibility under the machine learning paradigm: when the agent is interrupted, it must not learn to prevent future interruptions.
- Introduction to Bayes' rule: Odds form
Bayes' rule is simple, if you think in terms of relative odds.
- Introduction to Logical Decision Theory for Analytic Philosophers
Why "choose as if controlling the logical output of your decision algorithm" is the most appealing candidate for the principle of rational choice.
- Introduction to Logical Decision Theory for Computer Scientists
'Logical decision theory' from a math/programming standpoint, including how two agents with mutual knowledge of each other's code can cooperate on the Prisoner's Dilemma.
- Introduction to Logical Decision Theory for Economists
An introduction to 'logical decision theory' and its implications for the Ultimatum Game, voting in elections, bargaining problems, and more.
- Introductory Bayesian problems
Bayesian problems to try to solve yourself, before beginning to learn about Bayes' rule.
- Intution pump
In philosophy, a metaphor or visualization used to shove the listener's intuition in a particular direction.
- Invisible background fallacies
Universal laws also apply to objects and ideas that may fade into the invisible background. Reasoning as if these laws didn't apply to less obtrusive concepts is a type of fallacy.
- Joint probability
The notation for writing the chance that both X and Y are true.
- Just a requisite
A tag for nodes that just act as part of Arbital's requisite system
- Known-algorithm non-self-improving agent
Possible advanced AIs that aren't self-modifying, aren't self-improving, and where we know and understand all the component algorithms.
- Laplace's Rule of Succession
Suppose you flip a coin with an unknown bias 30 times, and see 4 heads and 26 tails. The Rule of Succession says the next flip has a 5/32 chance of showing heads.
- Likelihood functions, p-values, and the replication crisis
What's the whole Bayesian-vs.-frequentist debate about?
- Limited AGI
Task-based AGIs don't need unlimited cognitive and material powers to carry out their Tasks; which means their powers can potentially be limited.
- Linguistic conventions in value alignment
How and why to use precise language and words with special meaning when talking about value alignment.
- Link glossary pages for overloaded words
If your subject is using what sound like ordinary-language words in a special sense, create a glossa…
- List of Eliezer's current most desired fixes and features
A place for Eliezer to note down his current list of personally-wanted features for editing and writing.
- List: value-alignment subjects
Bullet point list of core VAT subjects.
- Logical decision theories
Root page for topics on logical decision theory, with multiple intros for different audiences.
- Logical game
Game's mathematical structure at its purest form.
- Look where I'm pointing, not at my finger
When trying to communicate the concept "glove", getting the AGI to focus on "gloves" rather than "my user's decision to label something a glove" or "anything that depresses the glove-labeling button".
- Low impact
The open problem of having an AI carry out tasks in ways that cause minimum side effects and change as little of the rest of the universe as possible.
- Math 0
Are you not actively bad at math, nor traumatized about math?
- Math 1
Is math sometimes fun for you, and are you not anxious if you see a math puzzle you don't know how to solve?
- Math 2
Do you work with math on a fairly routine basis? Do you have little trouble grasping abstract structures and ideas?
- Math 3
Can you read the sort of things that professional mathematicians read, aka LaTeX formulas with a minimum of explanation?
- Mathematics
Mathematics is the study of numbers and other ideal objects that can be described by axioms.
- Mechanical Turk (example)
The 19th-century chess-playing automaton known as the Mechanical Turk actually had a human operator inside. People at the time had interesting thoughts about the possibility of mechanical chess.
- Meta tags
What are meta tags and when to use them?
- Meta-rules for (narrow) value learning are still unsolved
We don't currently know a simple meta-utility function that would take in observation of humans and spit out our true values, or even a good target for a Task AGI.
- Meta-utility function
Preference frameworks built out of simple utility functions, but where, e.g., the 'correct' utility function for a possible world depends on whether a button is pressed.
- Metaethics
Metaethics asks "What kind of stuff is goodness made of?" (or "How would we compute goodness?") rather than "Which particular policies or outcomes are good or not-good?"
- Methodology of foreseeable difficulties
Building a nice AI is likely to be hard enough, and contain enough gotchas that won't show up in the AI's early days, that we need to foresee problems coming in advance.
- Methodology of unbounded analysis
What we do and don't understand how to do, using unlimited computing power, is a critical distinction and important frontier.
- Mild optimization
An AGI which, if you ask it to paint one car pink, just paints one car pink and doesn't tile the universe with pink-painted cars, because it's not trying *that* hard to max out its car-painting score.
- Mind design space is wide
Imagine all human beings as one tiny dot inside a much vaster sphere of possibilities for "The space of minds in general." It is wiser to make claims about *some* minds than *all* minds.
- Mind projection fallacy
Uncertainty is in the mind, not in the environment; a blank map does not correspond to a blank territory. In general, the territory may have a different ontology from the map.
- Mindcrime
Might a machine intelligence contain vast numbers of unhappy conscious subprocesses?
- Mindcrime: Introduction
The more predictive accuracy we want from a model, the more detailed the model becomes. A very roug…
- Minimality principle
The first AGI ever built should save the world in a way that requires the least amount of the least dangerous cognition.
- Missing the weird alternative
People might systematically overlook "make tiny molecular smileyfaces" as a way of "producing smiles", because our brains automatically search for high-utility-to-us ways of "producing smiles".
- Modeling distant superintelligences
The several large problems that might occur if an AI starts to think about alien superintelligences.
- Moral hazards in AGI development
"Moral hazard" is when owners of an advanced AGI give in to the temptation to do things with it that the rest of us would regard as 'bad', like, say, declaring themselves God-Emperor.
- Moral uncertainty
A meta-utility function in which the utility function as usually considered, takes on different values in different possible worlds, potentially distinguishable by evidence.
- Most complex things are not very compressible
We can't *prove* it's impossible, but it would be *extremely surprising* to discover a 500-state Turing machine that output the exact text of "Romeo and Juliet".
- Multiple stage fallacy
You can make an arbitrary proposition sound very improbable by observing how it seemingly requires X, Y, and Z. This didn't work for Nate Silver forecasting the Trump nomination.
- Mutually exclusive and exhaustive
The condition needed for probabilities to sum to 1
- NGDP level targeting
Central banks ought to regularize the total flow of money to increase at a predictable 5% rate per year, and doing this would solve a surprising number of other problems.
- Natural language understanding of "right" will yield normativity
What will happen if you tell an advanced agent to do the "right" thing?
- Nearest unblocked strategy
If you patch an agent's preference framework to avoid an undesirable solution, what can you expect to happen?
- Newcomb's Problem
There are two boxes in front of you, Box A and Box B. You can take both boxes, or only Box B. Box A contains $1000. Box B contains $1,000,000 if and only if Omega predicted you'd take only Box B.
- Newcomblike decision problems
Decision problems in which your choice correlates with something other than its physical consequences (say, because somebody has predicted you very well) can do weird things to some decision theories.
- Niceness is the first line of defense
The *first* line of defense in dealing with any partially superhuman AI system advanced enough to possibly be dangerous is that it does not *want* to hurt you or defeat your safety measures.
- Nick Bostrom
Nick Bostrom, secretly the inventor of Friendly AI
- Nick Bostrom's book Superintelligence
The current best book-form introduction to AI alignment theory.
- No-Free-Lunch theorems are often irrelevant
There's often a theorem proving that some problem has no optimal answer across every possible world. But this may not matter, since the real world is a special case. (E.g., a low-entropy universe.)
- Non-adversarial principle
At no point in constructing an Artificial General Intelligence should we construct a computation that tries to hurt us, and then try to stop it from hurting us.
- Nonperson predicate
If we knew which computations were definitely not people, we could tell AIs which programs they were definitely allowed to compute.
- Normalization (probability)
That thingy we do to make sure our probabilities sum to 1, when they should sum to 1.
- Object-level vs. indirect goals
Difference between "give Alice the apple" and "give Alice what she wants".
- Odds
Odds express a relative probability.
- Omega (alien philosopher-troll)
The entity that sets up all those trolley problems. An alien philosopher/troll imbued with unlimited powers, excellent predictive ability, and very odd motives.
- Omnipotence test for AI safety
Would your AI produce disastrous outcomes if it suddenly gained omnipotence and omniscience? If so, why did you program something that *wants* to hurt you and is held back only by lacking the power?
- Ontology identification problem
How do we link an agent's utility function to its model of the world, when we don't know what that model will look like?
- Ontology identification problem: Technical tutorial
Technical tutorial for ontology identification problem.
- Open subproblems in aligning a Task-based AGI
Open research problems, especially ones we can model today, in building an AGI that can "paint all cars pink" without turning its future light cone into pink-painted cars.
- Optimization daemons
When you optimize something so hard that it crystalizes into an optimizer, like the way natural selection optimized apes so hard they turned into human-level intelligences
- Oracle
System designed to safely answer questions.
- Orthogonality Thesis
Will smart AIs automatically become benevolent, or automatically become hostile? Or do different AI designs imply different goals?
- Other-izing (wanted: new optimization idiom)
Maximization isn't possible for bounded agents, and satisficing doesn't seem like enough. What other kind of 'izing' might be good for realistic, bounded agents?
- Paperclip
A configuration of matter that we'd see as being worthless even from a very cosmopolitan perspective.
- Paperclip maximizer
This agent will not stop until the entire universe is filled with paperclips.
- Parfit's Hitchhiker
You are dying in the desert. A truck-driver who is very good at reading faces finds you, and offers to drive you into the city if you promise to pay $1,000 on arrival. You are a selfish rationalist.
- Patch resistance
One does not simply solve the value alignment problem.
- Path targeting
Don't say "We want this price to go up at 2%/year", say "We want this to be $1 in year 1, $1.02 in year 2, $1.04 in year 3" and don't change the rest of the path if you miss one year's target.
- Path: Insights from Bayesian updating
A learning-path placeholder page for insights derived from the Bayesian rule for updating beliefs.
- Path: Multiple angles on Bayes's Rule
A learning-path placeholder page for learning multiple angles on Bayes's Rule.
- People
A category for human beings!
- Perfect rolling sphere
If you don't understand something, start by assuming it's a perfect rolling sphere.
- Philosophy
A stub parent node to contain standard concepts, belonging to subfields of academic philosophy, that are being used elsewhere on Arbital.
- Pivotal event
Which types of AIs, if they work, can do things that drastically change the nature of the further game?
- Posterior probability
What we believe, after seeing the evidence and doing a Bayesian update.
- Preference framework
What's the thing an agent uses to compare its preferences?
- Principles in AI alignment
A 'principle' of AI alignment is a very general design goal like 'understand what the heck is going on inside the AI' that has informed a wide set of specific design proposals.
- Prior
A state of prior knowledge, before seeing information on a new problem. Potentially complicated.
- Prior probability
What we believed before seeing the evidence.
- Prisoner's Dilemma
You and an accomplice have been arrested. Both of you must decide, in isolation, whether to testify against the other prisoner--which subtracts one year from your sentence, and adds two to theirs.
- Probability
The degree to which someone believes something, measured on a scale from 0 to 1, allowing us to do math to it.
- Probability notation for Bayes' rule
The probability notation used in Bayesian reasoning
- Probability notation for Bayes' rule: Intro (Math 1)
How to read, and identify, the probabilities in Bayesian problems.
- Probability theory
The logic of science; coherence relations on quantitative degrees of belief.
- Problem of fully updated deference
Why moral uncertainty doesn't stop an AI from defending its off-switch.
- Programmer
Who is building these advanced agents?
- Programmer deception
Programmer deception is when the AI's decision process leads it to optimize for an instrumental goal…
- Proof of Bayes' rule
Proofs of Bayes' rule, with graphics
- Proof of Bayes' rule: Intro
Proof of Bayes' rule, assuming you know the rule itself, and the notations for the quantities involved.
- Proving too much
If your argument could just as naturally be used to prove that Bigfoot exists and that Peano arithmetic is inconsistent, maybe it's an untrustworthy kind of argument.
- Psychologizing
It's sometimes important to consider how other people might be led into error. But psychoanalyzing them is also dangerous! Arbital discussion norms say to explicitly note this as "psychologizing".
- Querying the AGI user
Postulating that an advanced agent will check something with its user, probably comes with some standard issues and gotchas (e.g., prioritizing what to query, not manipulating the user, etc etc).
- Random utility function
A 'random' utility function is one chosen at random according to some simple probability measure (e.g. weight by Kolmorogov complexity) on a logical space of formal utility functions.
- Rationality
The subject domain for [ epistemic] and [ instrumental] rationality.
- Real-world domain
Some AIs play chess, some AIs play Go, some AIs drive cars. These different 'domains' present different options. All of reality, in all its messy entanglement, is the 'real-world domain'.
- Realistic (Math 1)
Real-life examples of Bayesian reasoning
- Reflective consistency
A decision system is reflectively consistent if it can approve of itself, or approve the construction of similar decision systems (as well as perhaps approving other decision systems too).
- Reflective stability
Wanting to think the way you currently think, building other agents and self-modifications that think the same way.
- Reflectively consistent degree of freedom
When an instrumentally efficient, self-modifying AI can be like X or like X' in such a way that X wants to be X and X' wants to be X', that's a reflectively consistent degree of freedom.
- Relative likelihood
How relatively likely an observation is, given two or more hypotheses, determines the strength and direction of evidence.
- Relevant limited AI
Can we have a limited AI, that's nonetheless relevant?
- Relevant powerful agent
An agent is relevant if it completely changes the course of history.
- Relevant powerful agents will be highly optimized
The probability that an agent that is cognitively powerful enough to be relevant to existential outc…
- Rescuing the utility function
If your utility function values 'heat', and then you discover to your horror that there's no ontologically basic heat, switch to valuing disordered kinetic energy. Likewise 'free will' or 'people'.
- Researchers in value alignment theory
Who's working full-time in value alignment theory?
- Rich domain
A domain is 'rich', relative to our own intelligence, to the extent that (1) its [ search space] is …
- Safe but useless
Sometimes, at the end of locking down your AI so that it seems extremely safe, you'll end up with an AI that can't be used to do anything interesting.
- Safe impact measure
What can we measure to make sure an agent is acting in a safe manner?
- Safe plan identification and verification
On a particular task or problem, the issue of how to communicate to the AGI what you want it to do and all the things you don't want it to do.
- Separation from hyperexistential risk
The AI should be widely separated in the design space from any AI that would constitute a "hyperexistential risk" (anything worse than death).
- Show me what you've broken
To demonstrate competence at computer security, or AI alignment, think in terms of breaking proposals and finding technically demonstrable flaws in them.
- Shutdown problem
How to build an AGI that lets you shut it down, despite the obvious fact that this will interfere with whatever the AGI's goals are.
- Shutdown utility function
A special case of a low-impact utility function where you just want the AGI to switch itself off harmlessly (and not create subagents to make absolutely sure it stays off, etcetera).
- Solomonoff induction
A simple way to superintelligently predict sequences of data, given unlimited computing power.
- Solomonoff induction: Intro Dialogue (Math 2)
An introduction to Solomonoff induction for the unfamiliar reader who isn't bad at math
- Some computations are people
It's possible to have a conscious person being simulated inside a computer or other substrate.
- Standard agent properties
What's a Standard Agent, and what can it do?
- Start
This page gives a basic overview of the topic, but may be missing important information or have stylistic issues. If you're able to, please help expand or improve it!
- Still needs work
The next step up from "Work in Progress". The page can be read as complete, but is a draft that needs further review and fine-tuning.
- Strained argument
A phenomenological feeling associated with a step of reasoning going from X to Y where it feels like…
- Strategic AGI typology
What broad types of advanced AIs, corresponding to which strategic scenarios, might it be possible or wise to create?
- Strength of Bayesian evidence
From a Bayesian standpoint, the strength of evidence can be identified with its likelihood ratio.
- Strictly confused
A hypothesis is strictly confused by the raw data, if the hypothesis did much worse in predicting it than the hypothesis itself expected.
- Strictly factual question
A "question of strict fact" is one which is true or false about the material universe (and maybe some math) without introducing any issues of values, perspectives, etcetera.
- Strong cognitive uncontainability
An advanced agent can win in ways humans can't understand in advance.
- Stub
This page only gives a very brief overview of the topic. If you're able to, please help expand or improve it!
- Subjective probability
Probability is in the mind, not in the environment. If you don't know whether a coin came up heads or tails, that's a fact about you, not a fact about the coin.
- Sufficiently advanced Artificial Intelligence
'Sufficiently advanced Artificial Intelligences' are AIs with enough 'advanced agent properties' that we start needing to do 'AI alignment' to them.
- Sufficiently optimized agents appear coherent
If you could think as well as a superintelligence, you'd be at least that smart yourself.
- Superintelligent
A "superintelligence" is strongly superhuman (strictly higher-performing than any and all humans) on every cognitive problem.
- Task (AI goal)
When building the first AGIs, it may be wiser to assign them only goals that are bounded in space and time, and can be satisfied by bounded efforts.
- Task identification problem
If you have a task-based AGI (Genie) then how do you pinpoint exactly what you want it to do (and not do)?
- Task-directed AGI
An advanced AI that's meant to pursue a series of limited-scope goals given it by the user. In Bostrom's terminology, a Genie.
- Terminal versus instrumental goals / values / preferences
Distinguish events wanted for their consequences, from events wanted locally.
- The AI must tolerate your safety measures
A corollary of the nonadversarial principle is that "The AI must tolerate your safety measures."
- The Robots, AI, and Unemployment Anti-FAQ
Q. Are the current high levels of unemployment being caused by advances in Artificial Intelligence …
- The empiricist-theorist false dichotomy
No discussion here yet: See https://www.facebook.com/groups/674486385982694/permalink/7846641016315…
- The rocket alignment problem
If people talked about the problem of space travel the way they talked about AI...
- Theory of (advanced) agents
One of the research subproblems of building powerful nice AIs, is the theory of (sufficiently advanced) minds in general.
- Tiling agents theory
The theory of self-modifying agents that build successors that are very similar to themselves, like repeating tiles on a tesselated plane.
- Time-machine metaphor for efficient agents
Don't imagine a paperclip maximizer as a mind. Imagine it as a time machine that always spits out the output leading to the greatest number of future paperclips.
- Total alignment
We say that an advanced AI is "totally aligned" when it knows *exactly* which outcomes and plans are beneficial, with no further user input.
- Toxoplasmosis dilemma
A parasitic infection, carried by cats, may make humans enjoy petting cats more. A kitten, now in front of you, isn't infected. But if you *want* to pet it, you may already be infected. Do you?
- Transparent Newcomb's Problem
Omega has left behind a transparent Box A containing $1000, and a transparent Box B containing $1,000,000 or nothing. Box B is full iff Omega thinks you one-box on seeing a full Box B.
- True Prisoner's Dilemma
A scenario that would reproduce the ideal payoff matrix of the Prisoner's Dilemma about human beings who care about their public reputation and each other.
- Ultimatum Game
A Proposer decides how to split $10 between themselves and the Responder. The Responder can take what is offered, or refuse, in which case both parties get nothing.
- Underestimating complexity of value because goodness feels like a simple property
When you just want to yell at the AI, "Just do normal high-value X, dammit, not weird low-value X!" and that 'high versus low value' boundary is way more complicated than your brain wants to think.
- Understandability principle
The more you understand what the heck is going on inside your AI, the safer you are.
- Unforeseen maximum
When you tell AI to produce world peace and it kills everyone. (Okay, some SF writers saw that one coming.)
- Universal prior
A "universal prior" is a probability distribution containing *all* the hypotheses, for some reasonable meaning of "all". E.g., "every possible computer program that computes probabilities".
- Unphysically large finite computer
The imaginary box required to run programs that require impossibly large, but finite, amounts of computing power.
- Updateless decision theories
Decision theories that maximize their policies (mappings from sense inputs to actions), rather than using their sense inputs to update their beliefs and then selecting actions.
- User manipulation
If not otherwise averted, many of an AGI's desired outcomes are likely to interact with users and hence imply an incentive to manipulate users.
- User maximization
A sub-principle of avoiding user manipulation - if you see an argmax over X or 'optimize X' instruction and X includes a user interaction, you've just told the AI to optimize the user.
- Utility
What is "utility" in the context of Value Alignment Theory?
- Utility function
The only coherent way of wanting things is to assign consistent relative scores to outcomes.
- Utility indifference
How can we make an AI indifferent to whether we press a button that changes its goals?
- Valley of Dangerous Complacency
When the AGI works often enough that you let down your guard, but it still has bugs. Imagine a robotic car that almost always steers perfectly, but sometimes heads off a cliff.
- Value
The word 'value' in the phrase 'value alignment' is a metasyntactic variable that indicates the speaker's future goals for intelligent life.
- Value achievement dilemma
How can Earth-originating intelligent life achieve most of its potential value, whether by AI or otherwise?
- Value alignment problem
You want to build an advanced AI with the right values... but how?
- Value identification problem
The subproblem category of value alignment which deals with pinpointing valuable outcomes to an adva…
- Value-laden
Cure cancer, but avoid any bad side effects? Categorizing "bad side effects" requires knowing what's "bad". If an agent needs to load complex human goals to evaluate something, it's "value-laden".
- Vinge's Law
You can't predict exactly what someone smarter than you would do, because if you could, you'd be that smart yourself.
- Vinge's Principle
An agent building another agent must usually approve its design without knowing the agent's exact policy choices.
- Vingean reflection
The problem of thinking about your future self when it's smarter than you.
- Vingean uncertainty
You can't predict the exact actions of an agent smarter than you - so is there anything you _can_ say about them?
- Wants to get straight to Bayes
A simple requisite page to mark whether the user has selected wanting to get straight into Bayes on …
- Waterfall diagram
Visualizing Bayes' rule as the mixing of probability streams.
- Waterfall diagrams and relative odds
A way to visualize Bayes' rule that yields an easier way to solve some problems
- Well-calibrated probabilities
Even if you're fairly ignorant, you can still strive to ensure that when you say "70% probability", it's true 70% of the time.
- William Frankena's list of terminal values
Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions...
- Work in progress
This page is being actively worked on by an editor. Check with them before making major changes.
- You can't get more paperclips that way
Most arguments that "A paperclip maximizer could get more paperclips by (doing nice things)" are flawed.
- You can't get the coffee if you're dead
An AI given the goal of 'get the coffee' can't achieve that goal if it has been turned off; so even an AI whose goal is just to fetch the coffee may try to avert a shutdown button being pressed.
- Zermelo-Fraenkel provability oracle
We might be able to build a system that can safely inform us that a theorem has a proof in set theory, but we can't see how to use that capability to save the world.
>As you've probably gathered, I feel hopeless about case (1).
Okay, I didn't understand this. My …
"Prior probability" doesn't rely on Bayes's Theorem, but the notion of a Bayesian prior does - it's …
(Should I be replacing 'approval-directed' with 'act-based' in my future writing?)
The intended mea…
- (2) Do something other than quietly making the domain of every page I create be 'Eliezer Yudkowsky…
- (3) I seem to not have an option to leave Editor Comments if I own the page, or my window is too s…
2^100 + 2^99 + 2^98... + 1 = 2^101 - 1.
> Are there meaningful policy differences between different shades of case (2)?
If all of our unce…
> Are you asking for safety even if one of these systems or subsystems becomes omniscient while othe…
> Consider the first AI system that can reasonably predict your answers to questions of the form "Mi…
> Do we disagree about this point? That is, do you think that such a pseudo-genie would predict me i…
> Here is my understanding of Eliezer's picture (translated into my worldview): we might be able to …
> It seems like the only advantage of the genie is that it doesn't make prediction errors about huma…
> Lorem
A whole sentence!
> My weak claim is that the pseudo-genie will not have catastrophic failures unless either (1) it ma…
> So you see the difference as whether the programmers have to actually supply the short-term object…
> The key property we want from the distinguisher is that it can learn to detect relevant difference…
> The obvious patch is for a sufficiently sophisticated system to have preferences over its own beha…
> To the extent we can set up all of these problems as parts of a learning problem, it just seems li…
> We don't have to explicitly cover injunctions, just to provide information that allows the agent t…
Added:
- (1) Make greenlinks in mobile popups followable.
- There's no reasonable way for a user t…
As regards 4, I'd say that while there may *theoretically* be arbitrarily powerful agents in math-sp…
By a "meta solution" I meant, e.g., coherent extrapolated volition, or having an AI that can detect …
Can we properly classify this as an error? If there's an AI that will be hacked, or maybe hack itse…
Darn it, I wanted to use this term to distinguish "not-explictly-consequentialistically optimizing f…
I agree this page is problematic in present form and probably needs to be rewritten by Rob Bensinger…
I don't understand what you mean. In computer security generally, breaking an existing system, espe…
I doubt it will satisfy you, but see the added "Selfish bastards" and "Why include everyone" section…
I have similar qualms about the name. Got something better?
Leaving that aside, if you have an AI …
I think I once saw either Andrew Gelman or Carl Shulman do the "there is an incredibly small chance …
I think my intent was something like, "lowercase things are simple concepts, capitalized things are …
I think one will often still need 'introductory' or 'tutorial' type pages that walk through the hier…
I think that many readers will have an easier time imagining 'what we can do by knowing a theorem is…
I think we have a foundational disagreement here about to what extent saying "Oh, the AI will just p…
I think we're going to have to specialize the terminology so we have separate words for "learn any g…
I was trying to say "append 1 to previous sequence". I guess it needs explanation.
I'll edit to be more precise: A CDT agent thinks "me and an LDT agent facing off against 99 other LD…
I'm not quite sure what claim of mine you're critiquing; can you spell out explicitly what you think…
I'm still not clear on what you think is false / what you think is the reality. Computer security a…
If you dump enough computing power into hill-climbing optimization, within a Turing-general policy s…
If you're trying to build your preference framework *using* induction to learn, e.g., what a 'user' …
In case a new user is confused by hovering a green link and seeing the popup suddenly poof in; in th…
It's 12 + 3 + 1. I'll edit to make clearer, but your comment exposed a bug in our LaTeX parsing so …
It's not obvious to me that these two approaches mean the same thing. Let's say that an AI sees som…
Just to not leave you completely dangling here, how about utility indifference?
K, will modify going forward.
Makes sense (though the versus you quote wasn't being advocated as a fair example by either agent). …
No individual compressor can monotonically decrease file sizes.
And we count the string of .rar.7z.…
Oh my God you don't know about instrumentally convergent corrigibility incorrigibility
How could I …
Paul, I didn't say "99%" lightly, obviously. And that makes me worried that we're not talking about…
Paul, I don't disagree that we want the AI to think whatever thought it ought to think. I'm proposi…
Paul, I'm having trouble isolating a background proposition on which we could more sharply disagree.…
Questions like these seem to me to have obvious unbounded formulations. If we're talking about a mo…
Should we really be lorem-ing?
Test 1
Test 2
Test 3
Test 4
The concern is for when you have a preference-limited AI that already contains enough computing powe…
The definition of 'relevant & limited' seems sensitive to beliefs about fast vs. slow takeoff, check…
The key question is not whether particular industries get automated. They will be. But so was weav…
The point of 'efficiency' is that:
- It's an extremely plausible thing to expect given enough raw c…
There's 6 successively stronger arguments listed under "Arguments" in the current version of the pag…
To make sure we're on the same page, Orthogonality is true if it's possible for a paperclip maximize…
Well, the *purpose* is to avoid the AGI classifying potential goal fulfillments in a way that, from …
Yes. That includes both the case where the length is specified outside the program, and the case wh…
question
- Do we need to worry about AI?
Why worry about AI when nothing has gone wrong yet?
wiki
- A Return to Discussion
- A clarification period for claims is net positive for Arbital
Example pros: Claims are more carefully defined and less ambiguous, less wrong questions visible
Ex…
- A whirlwind tour
A rapid tour of Eric's thoughts on the accelerator project.
- A-Class
This page is well-written, high-quality, and essentially complete.
- Accelerator FAQ
## Project FAQ
### What is this all about?
A large number of people are driven to move the needle…
- Accelerator Project
The Accelerator Project aims to create a low-cost environment which facilitates rapid personal growt…
- Accelerator project
Redirect: Accelerator Project
- An Effective Altruist's Guide to the Future
The future is unknown, but not unknowable. What evidence do we have about what will happen, and what considerations are most important for doing good in a rapidly changing world?
- Arbital Slack
Where the cool kids hang out.
- Arbital arbiter
Arbiters provide oversight and dispute resolution to an Arbital domain.
- Arbital archive
An archive of all public content on Arbital!
- Arbital claims are significantly more useful* when they are fairly well-specified and unambiguous**
\* At least 30% more valuable to people sharing models.
** Not lojban level, but with some thoug…
- Arbital completed project
Completed projects are essentially finished, having achieved the goals they set out to.
- Arbital content request
Arbital doesn't explain something you'd like to learn? We'd like to know, so we can prioritize.
- Arbital draft
Drafts are private work-in-progress pages.
- Arbital examplar pages
Exemplar pages on Arbital.
- Arbital external resources
Arbital wants to link users to great content, wherever it is!
- Arbital featured project
Featured projects are active, currently promoted projects.
- Arbital listed page
Listed pages have been accepted into at least one domain and checked by a reviewer.
- Arbital markdown demo
Demo of Arbital's markdown
- Arbital math levels
How mathy do you like your pages?
- Arbital ongoing project
Ongoing projects are not complete, but also not high activity enough to promote.
- Arbital project outline
Project outlines are in-progress proposals for projects.
- Arbital projects
Arbital projects are small-scale drives to fill in areas of content.
- Arbital quality
Arbital's system for tracking page quality.
- Arbital reviewer
Reviewers help writers improve their pages, check over all changes to Arbital's content, and assess page quality.
- Arbital scope
What kind of content is Arbital looking for?
- Arbital todo
So many things todo!
- Arbital trusted user
Trusted users can edit most pages directly, and don't need approval to add pages to a domain.
- Arbital user groups
Users can attain different powers and responsibilities on Arbital.
- Arbital's next pillar: Discussion
Arbital has a prototype discussion platform. If you're excited about this and want to participate, y…
- Arbital: Do what works
When deciding things on Arbital, think about the real goals, and move towards them.
- Arbital: Do what works: Justification
**Do what works** is meant to create a firm but flexible foundation for Arbital policy. The aims inc…
- Axiom
An **axiom** of a [theory\_mathematics theory] $T$ is a [well\_formed well-formed] [sentence\_mathem…
- B-Class
This page is mostly complete and without major problems, but has not had detailed feedback from the target audience and reviewers.
- Browser optimization for autodidacts
Your browser is your window on the 'net. Make it awesome.
- Building an intellectual edifice requires ongoing conversation
- C-Class
This page has substantial content, but may not thoroughly cover the topic, may not meet style and prose standards, or may not explain the concept in a way the target audience will reliably understand.
- Canons (What are they good for?)
- Chesterton's fence
If someone did something, it's generally good to understand their reasons for doing it before undoing it.
- Civilization scale energy
What are the main options for powering civilization, and how do they compare?
- Contributing to Arbital
Want to help Arbital become awesome?
- Convex
**Placeholder**
- Disambiguation
Several distinct concepts use this page's name, this page helps readers find what they're looking for.
- Element
**Placeholder**
- Eliezer's vision for Arbital
Why are we building this? What's the goal?
- Exercise
Meta tag for pages with exercises to practice a concept.
- External resources
This lens links out to other great resources across the web.
- Featured
This page is has been selected by Arbital to be featured and promoted.
- Featured math content
Some Arbital pages we think are great!
- Formal definition
This page gives a purely formal definition of a topic, rather than motivating, explaining, and giving examples.
- Formatting issues
This page has formatting or mathjax issues.
- Funding governance filtering
Current-best guess at how these parts of organization will happen.
### Funding
Eric provided initi…
- Grid scale storage
Scalable energy storage is required if civilization's switches to primarily renewables in order to keep the grid powered at night. What are the options and how do they compare?
- Guide
Meta tag for the start page of a multi-page guide.
- Hexayurt
External resource: http://hexayurt.com/. Feel free to add it!
- Needs examples
This page would benefit from more examples of the concept it teaches.
- Needs image
Meta tag for pages which would be improved by having images
- Needs links
This page could do with more greenlinks.
- Needs parent
This page is not attached to an appropriate parent page. If you know where it should go, please help categorize it!
- Needs requisites
This page has important requisites which are not listed. If you know what they are, you could help add them!
- Needs splitting by mastery
This page seems to serve multiple different mastery levels, and may benefit from being split into separate lenses.
- Needs technical summary
Meta tag for pages which need a technical summary.
- Needs work
Meta tag for pages which need content improvement.
- Opinion page
Opinion pages represent one position on a topic (often from a single author), and are not necessarily balanced or a reflection of consensus.
- Path to Acceleration
(Back to Accelerator home)
So, you know why we're doing this and what we're aiming for, a natural …
- Placeholder
This is an empty page created for structural reasons (parent, requisite, or teaches).
- Possible math pages
A list of things which we may want math pages on
- Proof
This page is just a proof, rather than a motivated explanation.
- Proof technique
**Placeholder**
- Proposed A-Class
Pages which have been proposed for A-Class status.
- Proposed B-Class
Pages which have been proposed for B-Class status.
- Real numbers are uncountable
The real numbers are uncountable.
- Resources and the future
Resource constraints are a widely held concern. Which are most likely to be limiting factors, and what can we do to relax those limits?
- Seed
Seeds are outlines of pages. They're not much use for readers, but can help authors.
- Sequence: Why Social Dynamics are So Complicated
Yesterday in NYC, we had a meetup delving into a few recent-ish blogposts about why Social Dynamics…
- Subjectivism vs Frequentism
Probability is in the mind, not in the environment. If you don't know whether a coin came up heads or tails, that's a fact about you, not a fact about the coin.
- Team Arbital
The people behind this site.
- The missing step between Zero and Hero
Creating a space for high potential people grow and improve the world at scale.
- Turing machine: External resources
* [Wikipedia](https://en.wikipedia.org/wiki/Turing_machine)
* [Wolfram MathWorld](http://mathworld.w…
- Unassessed
This page's quality has not been assessed.
- Who needs civilization?
The most common difference between my model and that of people I've discussed this with is that many…
- test123
test page
- testpage
test123
(1) Some concepts are "big concepts," in which case the main should give a high-level definition and…
(This part was written first, but the conclusion/outline seemed more valuable than this background t…
**Allow (slightly) more HTML**
This engine does allow a subset of HTML, but it's [extremely limited…
**Blog header**
Blog pages kind of feel like just normal pages. Having a header which is placed eit…
**Citations**
Making citations easy to create and maintain will cause more people to use them. Arbi…
**Feedback system / incentivising valuable criticism and productive response**
Partly inspired by […
**Import from [Mediawiki XML](https://www.mediawiki.org/xml/export-0.6.xsd)**
This would make movin…
**Karma system**
We want to keep track of several important characteristics separately. They are di…
**Navigation for the tree of information**
Implementing something like [MediaWiki's CategoryTree](h…
**Posted date on blog pages**
It's often important to know when a blog post was made. The originall…
**Pure Wiki Deletion (page blanking creates redlinks)**
Handling deletion on wikis is delicate, sin…
**Recent Changes / Edit Review system**
On traditional wikis the [recent changes page](https://wiki…
**Transclusion / Template system**
[Templates](https://www.mediawiki.org/wiki/Help:Templates) and […
**Uncategorized Pages Page**
This, combined with a good way of navigating tagged/categorized conten…
*nods*, definitely want to encourage discussion. However, people may not want their reasons for veto…
*nods*, it's definitely something we want, and an 80/20 version was on the last list of wiki feature…
*nods*, seems pretty similar to MW's CategoryTree? It's good to have, but it could be much more awes…
+1 for requiring feedback for downvoting. The original poster thought it was a good thing to post, a…
A bunch of specifics being pinned down would help. e.g. are the shelters inhabited, or just availabl…
Added, feel free to alter.
After discussion, allowing a few pre-defined classes and adding table to the whitelist would capture…
Ah, it's changed a bit, I'll update this page to reflect the new wording. The button labeled "join c…
Ah, okay, current arbital deletion already corresponds to this. Good :). (tests), deleting a page in…
Also, if you think the name is suboptimal, feel free to bring it up on the #math Slack and change un…
Approved, but the summary could do with a bit of improvement, make it something that a non-mathemati…
Consider using Arbital hidden text for the proof?
Cool, show me the designs/plans when that moves near the top of your list for feedback?
Could you add an intro/summary? It's good to have for both popups and avoiding having two headers at…
Discussions should be taggable with multiple domains, so the one place seems not quite right. Keep u…
Does "better" include "more like the in-group"? If yes, this seems very plausible. If no, I'd guess …
Done!
Encourage, not demand :), and maybe link to a blog post about why betting is good too?
Excellent point. Filing a bug.
For core/definition pages I think we want to have super modular content (easier browsing, lets peopl…
Good point. MathOverflow does have CS stuff. Changed.
Having a long redlink which does not point anywhere seems weird? Does the page it should point to no…
Having this as policy rather than something enforced by software seems better. I think Wikipedia did…
Hey, thanks for contributing! Would you like to visit our Slack channel to talk over your new pages?…
Hits good points, but awkwardly structured / worded in a few places. I can fix, but would reorganize…
I foresee good reputation systems being extremely valuable (essentially necessary to scale while mai…
I had been thinking of avoiding giving specifics in public, but the new Arbital pages does, so I'll …
I suspect if I were doing it I'd find it easier to structure and interlink with split first, but whi…
I think having it as a requisite is best? I see the issue, but some people may arrive from other pag…
I think the intro is likely to be confusing, and not give the info that a user arriving here wants? …
I want a clarification on the claim. How should this be handled, should it be attached to the claim?…
I'd add a once sentence "this is what it is / why it matters" thing. Perhaps something about efficie…
I'd lean towards mostly positives / integrating these two, rather than just negatives we want to avo…
I'd really want to tell this not as a whole new vision, but as moving onto a different part of an ex…
I'd tell this story fairly differently. This is not really how I saw math, and presenting it as not-…
I'll add a link when the archive exists, Alexei said to write the page first.
I'm skeptical of infinite nesting. It does help in some cases, but disrupts the vast bulk of threads…
I've added a +1 and link to this from the make tables possible bug. Currently, the workaround is to …
If applied to all "is a" statements, some pages are going to have a lot of parents eventually. This …
In both the notes, "recent pivot", I'd avoid pivot framing, to go with the whole "we're building the…
Is there any part of Arbital we can put as "hey, want to make this part?" so the community can help …
Is there no standard perspective that says:
Very few elections are decided by a single vote, but th…
It's in the same direction, yea. Even if relocating on earth captured all the wins (I would guess in…
Joe made a good point about the way this is phrased not sorting people quite right:
joe [11:50 AM] …
Link to a description of the library of babel?
Links to examples would go great here.
Looks like a mathjax error?
Making a page and greenlinking to it (with comments / edits / splits available) seems fine to me?
Maybe refer to the advantages of being compact over the unary system (all marks are worth one unit)?…
Maybe something other than æ would be better? like /////?
Maybe the alternate variants would be best on separate child pages, with links to them from this pag…
Mention them?
More believable, but risks discouraging people from writing the other types of content? I think ambi…
Neat, I'm a contrarian. I guess I should explain why my credence is about 80% different from everyon…
Needs some cushioning, to avoid setting expectations of not just powerful dictator-staff and arrogan…
Not clear what this means?
Not sure if it's high value to go into details, I'm unsure which parts help and it's not a set of th…
Note to self (or others): Add link from https://wiki.lesswrong.com/wiki/Decision_theory when this ex…
Note: I'm not certain about the alternate wording, and meant to suggest changes to the math 0 or mat…
Okay. I'm happy to be a sounding board/give input when you feel like this is one of the next steps.
…
Ops. link was still pointing to the wrong place. Fixed, thanks for reporting!
Or at the top as an expandable breadcrumb?
Perhaps make it a new replacement claim, and notify everyone who marked this claim that it's been re…
Perhaps this could be all in a note, with the only default-viewable thing saying we're financially s…
Perhaps, it currently feels ambiguous as to whether you're looking to use volunteers, or just saying…
Plausibly not, maybe having just one is better. Scanning through the summaries on /explore/math/ is …
Possibly have it hidden for logged-in users, but shown to logged out users? It'd be good for casual …
Reddit's reputation system gives new arrivals equal weight to long-standing highly trusted members o…
Repetition of scope in the note is mildly awkward.
Seems ill-fitting with the others, I'd drop this entirely. I doubt anyone who would aggressively exp…
Seems like greenlinking to the term gets you that, minus auto-suggest which seems like it'd get unwi…
Seems like there's main two parts:
**How do we make it intuitive for editors**
Displaying template…
Should we discourage titles for the top section of a page (e.g. Group homomorphism)? I think the top…
Something more humble will probably work better, "hope" is a good word here. Deprecated -> archived …
That's another approach, but gets into annoying subjective judgements really easily. I'm confident i…
The best discussion platform I know of currently is http://www.discourse.org by some of the people b…
The intro paragraph, the clickbait seems fine.
The notification showed me my post rather than the comment.
The structuring feels fairly awkward, I'd rewrite with high-value of X changed to something more hum…
There is now! This page even has a TOC.
This does not seem like it'd be transparent, esp. at math 0? The popover also seems potentially conf…
This gives a few clear examples, but does not help much with slightly less clear judgements (e.g. sh…
This is a cool page, but I think it (esp. the last paragraph) goes too fast for many math 1 readers,…
This is a great page! I think the intro/summary could be made a little more accessible though? The u…
This is a rough jumble of thoughts explaining some of the reasoning behind the proposed policy.
**D…
This page looks like it's going to be very large, perhaps splitting a bunch of parts out into childr…
This seems great for some domains (especially rapidly evolving scientific fields and many political/…
This sentence seems unclear/confusing?
True, there may be odd exceptions. And yep, fixed :)
Unclear how best to proceed. In particular, I'm unsure about how Proof, List, and Formal definition …
Which instance of that are you referring to (there are five)?
Working out how to handle karma on wikis is something I spent a bunch of brain cycles on a while ago…
Would Product (mathematics) be an appropriate name, or does category theory's use of the term point …
Would you like to come to our slack? If you send your email to eric@bruylant.com I'll invite you :)
Yep, that is what I meant. Create new page is less exposed now we've moved the Arbital math home. It…
Yep, there's at least high variability. Especially if the things it could be taken to mean are thing…
Yes in guides and explanations, I think yes universally but pinging slack for opinions.
You're welcome to join our slack channel if you'd like to get more real-time feedback, send me your …
Arbital is an online, collaborative platform for explaining <s>everything</s> math ([Arbital\_scope…
awkwardish, probably best drop functional, and maybe use network of knowledge rather than database?
eric_b [2:39 AM]
I'd add a "what is a 'number' anyway"-type page with an explanation of the genera…
eventual and unilateral don't really fit, and this sentence does not really make sense in general. C…
hm, changed it a bit, how's the new version?
hm, do you actually need that discussion? In no case does an agent know in advance that their vote w…
hmm.. I did parse that from it, it's maybe fine as-is. I think the thing is "whose" feeling personal…
kinda awkwardly worded, could pack more of a punch with some optimization.
so8res: "I would set up the page as follows:
A group homomorphism is X. Key properties of group hom…
soften or remove, especially the word dictate. facilitate perhaps?
soften, perhaps talk about encouraging good epistemic norms, give details/examples so people don't g…
question
wiki
- P (Polynomial Time Complexity Class)
P is the class of problems which can be solved by algorithms whose run time is bounded by a polynomial.
- Turing machine
A Turing Machine is a simple mathematical model of computation that is powerful enough to describe any computation a computer can do.
wiki
- A $1 donation to a top animal charity alleviates more suffering than is caused by a day of eating meat.
For the purposes of this claim, top animal welfare charities include:
- [Animal Charity Evaluators…
- A permanent, self-sustaining off-Earth colony would be a much more effective mitigation of x-risk than even an equally well funded system of disaster shelters on Earth.
See also the less precise claim: Establishing a permanent off-Earth colony would be a useful way to …
- Another playpen child
asdf
- Arbital page alias
The alias is a short, unique name assigned to each page. For example: "arbital_alias".
The alias u…
- Arbital page clickbait
The text you are reading right now is clickbait.
- Arbital page title
The title of a page is shown at the top of the page (e.g. "Arbital page title") and in most places …
- Consciousness research and valence research are tractable
See: Principia Qualia: blueprint for a new cause area, consciousness research with an eye toward eth…
- Consciousness research is critically important
See: Principia Qualia: blueprint for a new cause area, consciousness research with an eye toward et…
- Consciousness research is time-sensitive
See: Principia Qualia: blueprint for a new cause area, consciousness research with an eye toward et…
- Crowdsourcing moderation without sacrificing quality
- Discussion norms
What makes conversation productive?
- Donor coordination
How should donors behave, given that their actions may affect other donors?
- Double Crux — A Strategy for Resolving Disagreement
- Effective altruism
What's the most good you can do?
- Eliezer office hours
A series of meetings with Big Yud himself.
- Eliezer office hours 2016-04-19
### Steph's notes from the actual meeting:
- "good question" -> people upvoting your question is a g…
- Eliezer office hours 2016-05-07
##Questions for Eliezer##
###Strategy###
Q: What does a discussion of which EA cause to donate to …
- Establishing a permanent off-Earth colony would be a useful way to mitigate x-risk
- Posed by [purplepeople](http://effective-altruism.com/user/purplepeople/) on the [EA Forum](http:/…
- Ethics Offsets to the Rescue
Hate hurting animals, but love eating meat? Throw money at the problem!
- Ethics research should proceed in parallel to value alignment research
- Existential risk
> Because of accelerating technological progress, humankind may be rapidly approaching a critical ph…
- For mitigating AI x-risk, an off-Earth colony would be about as useful as a warm scarf
H/T to Eliezer Yudkowsky for ["warm scarf"](https://www.facebook.com/robert.wiblin/posts/75711126783…
- Growing the EA movement is net positive
- High-speed explanation
Use this tag to indicate that a page offers a relatively faster and more terse explanation.
Note th…
- If we can’t lie to others, we will lie to ourselves
- In order to have a shared conversation, it would help to have a Schelling set of posts.
- In order to think accumulatively we must have a shared conversation.
- Is Eric's music better than Steph's?
This is a sample debate topic.
- Low-speed explanation
Use this tag to indicate that a page offers a relatively slower, more gentle, or more wordy explanat…
- Meta (Arbital Labs)
Tag for meta discussion in Arbital Labs
- Needs motivation
A tag for text that could benefit from some motivating statements. Why is the reader interested in w…
- Omniment
Nothing here yet.]
Automatically generated page for "Omniment" group.
If you are the owner, click …
- Organizations' utility curves for money are usually smooth (fundraiser milestones notwithstanding)
- Our community has a shot at seriously helping with x-risk.
- Politics
Tag for political discussion in Arbital Labs
- Principia Qualia: blueprint for a new cause area, consciousness research with an eye toward ethics and x-risk
Claims discussed in this post:
- [claim([6xd])]
- [claim([6x4])]
- [claim([6xf])]
- Project outline: Intro to the Universal Property
Outline detailing all the work required for a proposed Arbital Project
- Project proposal: Intro to numbers
Should Arbital's first "project" be a guide to numbers?
- The Necessity of Credibility
- The ideal Arbital math page
Think of the best math textbook you've ever read -- why was it good?
- The world needs something like a miracle of good thought if it is to have decent odds of overcoming x-risk.
- Thinking accumulatively is one of the keys to solving existential risks.
- We do not currently have a single shared conversation.
- fdsa
Nothing here yet.]
Automatically generated page for "fdsa" group.
If you are the owner, click [her…
"has some resistance to Eternal September" -> "is resistant to Eternal September" ?
"speed have been" -> "speed will have been" ?
$10
***what*** is **bold** *italics*
> Can you state explicitly what background assumption would lead you to think that an AI which behav…
> I agree that there are some x-risks (like global warming) that are helped by a colony, but most ar…
> I'm pushing back a little on this "classifier that avoids false positives" description because tha…
> If Wisconsin is trading cheese with Ohio, and then Michigan becomes much better at producing chees…
> This is silly
Perhaps
> then you ought to focus predominantly on something else
This does not s…
> describes underlying sets specifically in terms of algebraic structures
Yeah, I was wondering abo…
> is utility the donor gets from donating money "smooth" with respect to the amount raised
Ideally …
> reinforcing their view that sophisticated AI systems will be agents persuing explicitly represente…
A probability doesn't seem like the right way to measure this.
Above, first-person pronouns referred to Player 2, but now they seem to refer to Player 1. Was the s…
Actually, Kasparov won the 1996 match with a score of 4-2. This marked the first time a computer pro…
Ah, one additional thing I'm confused about -- what do $X_i$ and $x_i$ refer to? I thought $X_i$ ref…
And is there any significance to the fact that A and -A are divided by a straight line, but B and -B…
Are all the words in the free group, or just the freely-reduced words?
If the latter, does saying t…
Are the *other* tags part of the quality scale?
When a page has the message, "This page's quality i…
Are the them's in this sentence referring to different things? My guess is we're rescuing the theori…
Are there existing pages that need this tag?
I'm wondering if adding a second, brief summary is the…
Are we more believable if we narrow the scope of our ambitions?
Consider, "best place online to fin…
Aren't programs for Turing machines specified as marks on an infinite tape?
I was interpreting 100-…
Better?
But I do think it is a good question.
But wouldn't following that principle lead you to say the codomain is the positive reals, since that…
Can we just replace the following:
> According to the frequentist interpretation, the model is sayi…
Can you expand on what you mean here? They're a higher priority in what sense?
Improving them is a …
Confused about the role this clause is playing. What does "which" refer to --
the proportional form…
Current thinking is that we should allow claims to be edited, but that past users' votes appear gray…
Did you just swap the pronouns here? In the previous sentences the speaker was the seller and the li…
Did you mean this to link to How many bits to a trit??
Did you mean to use "caloric" instead of "caloric fluid" here (and many following places)? I keep re…
Do the different biases of coin correspond to different effect sizes? (E.g. large effect corresponds…
Do we want *citation needed* norms on Arbital?
(At a higher level, do we want readers to be able to…
Does this actually work for any proportions of A and B? Is there a simple proof?
Does this make the definition of the codomain somewhat arbitrary?
The squares of reals happen to be…
Does this mean that it's enforced internally that the same alias can not be a page alias and a quest…
Does x correspond to a *statement* (as used in the previous sentence about expressiveness), or does …
Eric's notes:
- modes
- reply to comments mode
- maintain my pages mode
- marks, todos, etc.…
Even if we expect to implement an [indirect](https://en.wikipedia.org/wiki/AI_control_problem#Indire…
Fair to paraphrase as: donor-as-silent-partner?
First use of "we" should indicate who "we" are, e.g. "We at Arbital..."
For counterpoint, see: http://effective-altruism.com/ea/ry/ethical_offsetting_is_antithetical_to_ea/…
For readers who just skimmed over $2^{3,000,000,000,000}$, and didn't parse it as "two to the three …
For reference, Lewis Bollard [estimates](http://www.openphilanthropy.org/blog/initial-grants-support…
From the [FB thread](https://www.facebook.com/robert.wiblin/posts/757111267835?comment_id=7571934182…
From the summary:
> the model is saying that there are a whole bunch of different places where some…
Here's another comment.
Hmm...
1) It seems weird to say that the model claims that there are a bunch of other models/events…
How about, "because I'm going to need six 10-digits to get up to a million, and something more than …
How is this archive accessed?
I agree that For mitigating AI x-risk, an off-Earth colony would be about as useful as a warm scarf.…
I am a real comment. Don't delete me please!
I did it.
I don't usually have this concern because I assume that the utility from extra money for an organiza…
I expect the tone of these last two sentences to rub some readers the wrong way. Would much be lost …
I find this sentence hard to read. Maybe the punctuation marks are in the wrong places?
I got lost here (and in the following equations). I think it's a combination of needing the "factori…
I got lost here -- I feel like I sort of know what "under permutation" means, but can't picture what…
I liked this explanation. In particular, the *obvious hard way* vs *sneaky easy way* contrast caught…
I might rephrase this to, "initial target" so it's clear that it was intended as a step along the pa…
I might try to introduce these terms one at a time, and a bit more slowly -- the paragraph up to thi…
I might write this as, "whereas, when multiplying 1 by 10 to get to x, you might have to multiply by…
I really like this domino analogy.
Also, I'd expect to see the word "all" somewhere in this first p…
I support the creation of a poset-office, but it's gotta be about posets!
I think *isomorphic* is too advanced vocabulary to be assumed for Math 1. Would this be a good oppor…
I think I'm happy to have TCS concepts on relatively equal footing with other math pages. Are there …
I think it would be worthwhile to explicitly call out that what's happening here is that we're repla…
I think this bit is slightly confusing. Is it a new lens per resource?
And what do you mean by stan…
I think this paragraph and the preceding paragraph would be great opportunities for hidden-text home…
I think this section conflates two things: 1) the role LW used to play, and 2) the role ultimate-Arb…
I think this should be a claim.
I think you may need to spell out this 10 times as many numbers part. This is a large unexplained st…
I went ahead and fixed the alias link.
I would add an "I assume" here in parentheses, so you're not putting words in their mouth, or projec…
I would consider leading with this question, to motivate the post.
I would expect this sentence only after another telling me that the observations were red car, honki…
I would just take "500 is using nearly 2.7 digits" out.
I would like to see an operationalization.
Who is our community? How many of us should move?
I'd be interested to know if you find yourself having that feeling a lot, while interacting with cla…
I'd take out 'young'
I'm having trouble parsing interpretation #1 -- which part is supposed to map onto the right hand si…
I'm not quite sure of this.
Suppose there are two different super-human chess AI's with different s…
I'm not sure I understand this part. Did you get "roughly 2" just by dividing 13 by 7?
Why should t…
I'm not sure what this equation is trying to tell me. Some parts of it are only true if A and B are …
I'm not sure what you mean about an exchange rate. Isn't a Pareto improvement something that makes e…
I'm reminded of this article: http://www.macroresilience.com/2011/12/29/people-make-poor-monitors-fo…
If these are included I think it would be good to also include explanations of why each one is wrong…
In other words, promoting this claim as worded, is misleading?
In this sentence I think you meant 'codomain' where you said 'domain' and 'image' where you said 'co…
Instead of telling me "It's OK if this doesn't make sense..." (No it's not okay! I was told this is …
Intro should be re-written so as not specific to algebraic structures.
Is it?
Is simplified Parfit's Hitchhiker the same as what was described above? I'm uncertain because this i…
Is the idea that a single organization should pursue X or Y and not worry about the fact that any gi…
Is there a difference between any particular 99-bit program and a 100-bit program with a 0 as the fi…
Is this sentence supposed to have a part saying, "and if we set the left-side red beam... starting i…
Is what follows the colon meant to be justification for what precedes it? I'm not following.
Isn't one coin and three dice better?
$2*3^6 = 432,$ and $\log_2(2) + 3*\log_2(6) \approx 8.75$
It seems that classifiers trained on adversarial examples may be finding (more) conservative concept…
It's good.
It's not totally clear what the antecedent of this "it's" is. (Because "it's" often means "it is the…
Just to make this super concrete, could we give some examples of topics we'd like to include later b…
May need to build the intuition that knowing how f(x) behaves tells me how f(c*x) is different from …
Maybe "gradual" would be a better term. I mean that there aren't sharp transitions where e.g. raisin…
Nice!
Nm, the longer explanation later in the page answered my question.
Not 2^100?
Not currently.
Not sure if it makes sense for this one to be a probability bar.
Here's an alternate version with a…
Note that PredictIt [currently thinks](https://www.predictit.org/Contract/4264/Will-the-next-US-pres…
Note that this is an admin-only feature for now. So maybe should be left off the style guide or mark…
Oh no! That's a bug, we are looking into it...
Oh, I thought it might be a pun. But nothing about the surrounding description sounded like a poset …
Okay, read through this section again, and I think it makes sense to me now. Would love to see an ex…
Omit the 'as'
Overall, I think the post covers most of the important points, but I think I'd want to cut some part…
Probably more like Math 2 or 2.5. I'm a programmer without any formal training in math beyond what's…
See also: http://www.metaculus.com/questions/377/will-donald-trump-be-the-president-of-the-united-st…
Seven tenths?
Should the p's and q's in one of these be switched?
So I suppose I should attempt a real reply.
I think:
- information hazards should be avoided
- peo…
Started having aha moments from here on down.
Still?
Suggest modifying the first clause (possibly by moving the 'only' from the second into the first) to…
Suppose there are existing generic techniques for developing classifiers that prioritize avoiding fa…
Testing out replies.
That's right. I know very little group theory.
I was just remarking to Stephanie how I was able to …
The page grows long. Perhaps it should be split into two or three pages?
The question of tradeoffs between X and Y and winners' curses reminds me of Bostrom's paper, [The Un…
The section sounds slightly too passive to me. What would you think of the following re-write?
Exte…
The user already knows they're on Arbital. Why not just call it "Guide" and "introductions"?
This is a clear explanation, but I think some formatting changes could enable readers to grok it eve…
This is helpful! I don't think I've seen a clearer description of the assumptions behind CEV or what…
This is one of the claims that Benjamin Hoffman made in his post, so I think we should leave the wor…
This is slightly confusing, because it's the first digit that's a 2.
This page is an outline for the Universal Property project.
Progress on the project will be measure…
This paragraph is good -- clearer than some of the other places where you tried to introduced this i…
This part is especially clear.
This should be a greenlink to a page where we explain what we mean by an "intuitive, multi-level exp…
This text is out of sync with the graphic -- the pic actually shows black tongue depressors.
This use of "naturally" may be jarring, since it may not feel obvious to the reader just being intro…
This wording suggests the group contains only some of the elements from the following list. I think …
Too Eliezer-voice. What would Sal Khan say?
Two L's
Wait, really? Is this a joke or does being transcendental follow from having to satisfy many constra…
We're going to feature whatever we choose as the current project on the front page, and I want to in…
What about calling this page the "tutorial" rather than "guide"? Tutorials are more likely to be int…
What do you mean by this -- could you say more about how it's different from the previous reason?
What's the difference between an operator and an operation?
Also what's the difference between an o…
When should we apply this tag?
Let's add some guidelines for when this tag is appropriate.
Eric Br…
Which calculation?
Whiteboard pics: https://plus.google.com/collection/kgEJNB
Why is it called a *decision problem*? As a reader looking for an intuitive understanding, that's on…
Why not just count explicitly?
I think the answer is, "because we want to teach what a bijection is…
Would be cool to have an image of an example graph here.
Would be great to have an example of the kind of formula one might expect to see.
Would it be appropriate to link to the Underlying set page here?
Would it be fair to summarize the idea of a conservative concept boundary as a classifier that avoid…
Would love to see some kind of footnote or popup that answers the question, "why not?"
Yeah, that wasn't a great comment from me :P
Yes, I think I should have used "question/objection" rather than comment. (But I'm trying do what fe…
Alexei Andreev can you say more about why you endorse this proposal? In particular, would you change…
Benjamin Hoffman Given your analysis, I'm surprised by your vote of 50%. You took what was given as …
[claim([6y8])]
and I *can* toss it?
asdf
asdf
asdffdsa
comma?
due to
fdsa
number systems
odd + odd doesn't equal even?
odds ratios?
The thing inside the log(this part) is an odds ratio, right?
output?
person
pick one
raises?
use colon instead?
wiki
- Formal Logic
Formal logic studies the form of correct arguments through rigorous and precise mathematical theories.
- Logic
Logic is the study of correct arguments.
wiki
- Phenomenological Complexity Classes
A phenomenological theory of differences in the complexity of minds-in-general.
- Solutions Paper - Solving Employment in the Light of Automation
GeM Labs solution space to tackle labour market challenges that arise from the rise of automation and artificial intelligence.
wiki
- Flexagon
Flexagons are flat models, that can be flexed in certain ways to reveal faces besides the two that were originally on the back and front
- Extensionality Axiom
If two sets have exactly the same members, then they are equal
wiki
- Exponential notation for function spaces
Why $Y^X$ is good notation for the space of maps from $X$ to $Y$
- Programming in Dependent Type Theory
Working with simple types in Lean
- Type theory
Modern foundations for formal mathematics.
wiki
- An introductory guide to modern logic
Logic, provability, Löb, Gödel and more!
- Church-Turing thesis
A thesis about computational models
- Church-Turing thesis: Evidence for the Church-Turing thesis
Why do we believe in CT thesis?
- Complexity theory
Study of the computational resources needed to compute something
- Complexity theory: Complexity zoo
Pass and see the exotic beasts coming from the lands of afar!
- Consistency
A consistent [-theory] is one in which there are well-formed statements that you cannot prove from i…
- Decision problem
Formalization of general problems
- Diagonal lemma
Constructing self-referential sentences
- First order linear equations
A **first order lineal equation** has the form
$$
u'=a(t)u+b(t)
$$
where $a$ and $b$ are continuous …
- Fixed point theorem of provability logic
Deal with those pesky self-referential sentences!
- Gödel II and Löb's theorem
[ Gödel's second incompleteness theorem] and [ Löb's theorem] are equivalent to each other. ]
The …
- Gödel's first incompleteness theorem
The theorem that destroyed Hilbert's program
- Kripke model
The semantics of modal logic
- Logical system
Logical systems (a.k.a. formal systems) are mathematical abstractions that aim to capture the notion…
- Löb's theorem
Löb's theorem
- Löb's theorem and computer programs
The close relationship between [ logic and computability] allows us to frame Löb's theorem in terms …
- Löbstacle
Imagine you have an artificial intelligence $D1$ who uses a logical [ knowledge base] and a certain …
- Modal combat
Modal combat
- Modal logic
The logic of boxes and bots.
- Modalized modal sentence
A [ modal sentence] $A$ is said to be **modalized** in $p$ if every occurrence of $p$ happens within…
- Morphism
A morphism is the abstract representation of a relation between mathematical objects.
Usually, it i…
- Natural number
The numbers we use to count: 0, 1, 2, 3, ...
- Normal system of provability logic
Between the modal systems of provability, the normal systems distinguish themselves by exhibiting ni…
- P vs NP
Is creativity purely mechanical?
- P vs NP: Arguments against P=NP
Why we believe P and NP are different
- Proof by contradiction
Discover what 'reductio ad absurdum' means!
- Proof of Gödel's first incompleteness theorem
##Weak form
The weak Gödel's first incompleteness theorem states that every [ $\omega$-consistent] […
- Proof of Löb's theorem
Proving that I am Santa Claus
- Provability logic
Learn how to reason about provability!
- Provability predicate
A provability predicate of a theory $T$ is a formula $P(x)$ with one free variable $x$ such that:
…
- Representability theorem for computable functions
A [ logical theory] $T$ is said to satisfy the **representability theorem for computable functions**…
- Rice's theorem and the Halting problem
We will show that Rice's theorem and the the halting problem are equivalent.
#The Halting theorem i…
- Solovay's theorems of arithmetical adequacy for GL
Using GL to reason about PA, and viceversa
- Standard provability predicate
Encoding provability as a statement of arithmetic
- Strong Church Turing thesis
A strengthening of the Church Turing thesis
- Teleological Measure and agency
A crackpot's research proposal
- Uncomputability
The diagonal function and the halting problem
- Without loss of generality
*Without loss of generality* (abbreviated as w.l.o.g.) is a common idiom in mathematics that remarks…
"Formula" and "Statement" can be interchanged freely.
Both refer to well-formed strings in the lang…
I do not think that the "Me facing off against 99 CDT agents and 1 LDT agent, versus an LDT agent fa…
I fail to see how this setup is not fair - but more importantly, I fail to see how LDT is losing in …
I love the effect, but I would drop one of the 'as much's for increased lyricism.
I think that the clickbait and the summary should be exchanged.
I'll try to cover probabilistic Turing machines in other article once I have time. Thanks for your i…
Might as well do it! No promises though.
My fault, it should be \ulcorner.
Out of curiosity, is there any reason you are avoiding calling this lemma by its traditional name, E…
Pedantic remark: Aren't you missing the identity of free groups in your intuitive construction?
We …
Probably you will want to precisely define **morphism** at some point, but I recommend you do it soo…
The problem I have in mind is deciding whether the Halting problem is equivalent to any statement of…
This is redacted in a very confusing way.
Yeah, that is the formal definition of the standard provability predicate. I'll add the page explain…
Alexei Andreev I was aiming for the Python syntax of the ternary operator, but I am still quite unfa…
2mn I am imagining myself in the shoes of somebody who doesn't know anything about cat theory gettin…
2mn I asked in Slack how can I grant you edit permisses, though the system doesnt seem to be in plac…
wiki
- Uncountability
Some infinities are bigger than others. Uncountable infinities are larger than countable infinities.
- Uncountability: Intro (Math 1)
Not all infinities are created equal. The infinity of real numbers is infinitely larger than the infinity of counting numbers.
- Uncountability: Intuitive Intro
Are all sizes of infinity the same? What does "the same" even mean here?
wiki
- Arguments
An argument is a formal reasoning, valid or not.
- Conjunctions and disjunctions
The fancy name for the "and" and "or" connectives.
- Law of syllogism
Deriving something from the conclusion of another thing.
- Modus tollens
Deriving a negation from another negation
- Negation of propositions
The proposition that is false if another one is true and vice-versa.
- Propositions
Propositions are statements with a truth value.
- Convex function
A function that only curves upward
- Convex set
A set that contains all line segments between points in the set
- Informed oversight
Incentivize a reinforcement learner that's less smart than you to accomplish some task
- Reliable prediction
How can we train predictors that reliably predict observable phenomena such as human behavior?
- Safe training procedures for human-imitators
How does one train a reinforcement learner to act like a human?
- Selective similarity metrics for imitation
Can we make human-imitators more efficient by scoring them more heavily on imitating the aspects of human behavior we care about more?
wiki
- Cardinality
The "size" of a set, or the "number of elements" that it has.
- Cauchy sequence
Infinite sequences whose terms get arbitrarily close together.
- Effective number of political parties
A way of quantifying the relative dominance of a few political parties regardless of their actual number.
- Exponential
Any function that constantly gets larger as a proportion of itself.
- Identity element
An element in a set with a binary operation that leaves every element unchanged when used as the other operand.
- Integers: Intro (Math 0)
The integers are the whole numbers extended into the negatives.
- Intro to Number Sets
An introduction to number sets for people who have no idea what a number set is.
- Irrational number
Real numbers that are not rational numbers
- Logistic function
A monotonic function from the real numbers to the open unit interval.
- Math 2 example statements
If you can read these formulas, you're in Math 2!
- Math 3 example statements
If you can read these formulas, you're in Math 3!
- Natural numbers: Intro to Number Sets
Natural numbers are the numbers we use to count in everyday life.
- Number
An abstract object that expresses quantity or value of some sort.
- Order of operations
Conventions used for disambiguating infix notation.
- Order relation
A way of determining which elements of a set come "before" or "after" other elements.
- Ordered field
An ordered ring with division.
- Proof that there are infinitely many primes
Suppose there were finitely many primes. Then consider the product of all the primes plus 1...
- Proportion
A representation of a value as a fraction or multiple of another value.
- Real number (as Dedekind cut)
A way to construct the real numbers that follows the intuition of filling in the gaps.
- The n-th root of m is either an integer or irrational
In other words, no power of a rational number that is not an integer is ever an integer.
- Totally ordered set
A set where all the elements can be compared as greater than or less than.
- Whole number
A term that can refer to three different sets of "numbers that are not fractions".
- Least common multiple
The **least common multiple (LCM)** of two positive natural numbers a, b is the smallest natural …
wiki
- The development of Artificial General Intelligence, as a scientific purpose for human life
Purpose here is not to be confused for teleological argument/theism/deities/subjective endeavours. Instead, this thread refers to teleonomy, or purpose in the realm of science/objectivity.
wiki
- Complete lattice
A poset that is closed under arbitrary joins and meets.
- Computer Programming Familiarity
Want to see programming analogies and applications in your math explanations? Mark this as known.
- Join and meet
Let $\langle P, \leq \rangle$ be a poset, and let $S \subseteq P$. The **join** of $S$ in $P$, deno…
- Join and meet: Examples
A union of sets and the least common multiple of a set of natural numbers can both be viewed as join…
- Join and meet: Exercises
Try these exercises to test your knowledge of joins and meets.
Tangled up
--------------------
!…
- Lattice (Order Theory)
A poset that is closed under binary joins and meets.
- Lattice: Examples
Here are some additional examples of lattices. $\newcommand{\nsubg}{\mathcal N \mbox{-} Sub~G}$
A f…
- Lattice: Exercises
Try these exercises to test your knowledge of lattices.
## Distributivity
Does the lattice meet op…
- Monotone function
An order-preserving map between posets.
- Monotone function: examples
Here are some examples of monotone functions.
A cunning plan
--------
There's a two-player game ca…
- Monotone function: exercises
Try these exercises and become a *deity* of monotonicity.
Monotone composition
-----
Let $P, Q$, …
- Order theory
The study of binary relations that are reflexive, transitive, and antisymmetic.
- Partially ordered set
A set endowed with a relation that is reflexive, transitive, and antisymmetric.
- Poset: Examples
The standard $\leq$ relation on integers, the $\subseteq$ relation on sets, and the $|$ (divisibilit…
- Poset: Exercises
Try these exercises to test your poset knowledge.
# Corporate Ladder
Imagine a company with five …
- Real analysis
The study of real numbers and real-valued functions.
- Relation
A **relation** is a set of [tuple\_mathematics tuples], all of which have the same [tuple\_arity ar…
About the citations: what I actually meant was that I want to have a bibliography, so that I can giv…
After another session of using Arbital, I have a few questions and comments.
1.) Is there any mecha…
Correct me if I'm wrong, but isn't it idiosyncratic to define $\leq$ as a predicate rather than a re…
I can't reproduce it either. Maybe it had something to do with the USB keyboard that I was using.
I have no plans to write about real analysis. I just created this page so that I could use it in con…
I intended this to be a wiki page. My plan is to gradually develop it into a full fledged tutorial o…
I see that there is a description of double scaling above. I assume that this is what "the product r…
I suspect that this is going to be too fast-paced for beginners. They are going to need multiple exa…
I think it's confusing to introduce multi-argument functions before talking about currying. This mak…
I think that every metric space is dense in itself. If X is a metric space, then a set E is dense in…
I think this is an informal presentation of a subject which should only be presented formally. There…
I understand what you're saying and I think it's a good point. The problem is that you're developing…
I'm pretty tired right now, but this definition seems kind of circular to me. It involves an infinit…
It looks like there is a word missing from this sentence. I'm not sure what it is trying to say.
Maybe. I haven't done so because the underlying set page describes underlying sets specifically in t…
Thanks Chris. Edit accepted.
The editor kept automatically scrolling to top when I was trying to edit this page in Firefox just n…
The title mentions Cauchy sequences, but the body does not. Doesn't this definition consider classes…
There is a page for linearly ordered set. It is called "totally ordered set". This is one of those s…
There is already a page about this topic, Join and meet.
This is sentence is kind of confusing. It seems like it's trying to say that if we know A = B, then …
This relies on a principle "other way" introduces but, in my opinion, is not explicit enough about: …
Yes, that's correct. I wonder if it is even a good idea to talk about transitive sets in the transit…
Alexei Andreev I was going to add an examples lens to this page, but I seem to have lost the ability…
Alexei Andreev This page includes a conditional example that only shows up for people who know real …
wiki
- Solving Intelligence
What does it mean exactly to "solve intelligence"?
- Examination through isomorphism
Isomorphism is the correct notion of equality between objects in a category. From the category-theor…
- Generalized element
A category-theoretic generalization of the notion of element of a set.
wiki
- Absolute Complement
The complement $A^\complement$ of a set $A$ is the set of all things that are not in $A$. Put simply…
- Antisymmetric relation
A binary relation where no two distinct elements are related in both directions
- Currying
Transforms a function of many arguments into a function into a function of a single argument
- Intersection
The intersection of two sets is the set of elements they have in common
- Operations in Set theory
An operation in set theory is a Function of two sets, that returns a set.
Common set operations inc…
- Relative complement
The relative complement of two sets $A$ and $B$, denoted $A \setminus B$, is the set of elements tha…
- Union
The union of two sets is the set of elements which are in one or the other, or both
wiki
- Binary notation
A way to write down numbers using powers of two.
- Boolean
A value in logic that evaluates to either "true" or "false".
- Modular arithmetic
Addition as traveling around a circle, instead of along a line.
wiki
- Malcolm Ocean's Arbitlog
Not actually a thing. Visit [malcolmocean.com](http://malcolmocean.com/).
wiki
- Axiom of Choice: Definition (Formal)
Mathematically speaking, what does the Axiom of Choice say?
- Axiom of Choice
The most controversial axiom of the 20th century.
- Axiom of Choice Definition (Intuitive)
Definition of the Axiom of Choice, without using heavy mathematical notation.
- Axiom of Choice: Guide
Learn about the most controversial axiom of the 20th century.
- Axiom of Choice: History and Controversy
Really? The *most* controversial axiom of the 20th century? Yes.
- Axiom of Choice: Introduction
What is the axiom of choice and why do I care?
- Bijective Function: Intro (Math 0)
Two boxes are bijective if they contain the same number of items.
- Category (mathematics)
A description of how a collection of mathematical objects are related to one another.
- Category theory
How mathematical objects are related to others in the same category.
- Cyclic Group Intro (Math 0)
A finite cyclic group is a little bit like a clock.
- Isomorphism
A morphism between two objects which describes how they are "essentially equivalent" for the purposes of the theory under consideration.
- Isomorphism: Intro (Math 0)
Things which are basically the same, except for some stuff you don't care about.
- Orbit-Stabiliser theorem: External Resources
External resources on the Orbit-Stabiliser theorem.
- Product (Category Theory)
How a product is characterized rather than how it's constructed
Cantor's argument also works in the finite case and this may serve to demonstrate the idea.
Conside…
Could be nice to add a concrete "real-life" (non-math) example, say like the following:
You are a d…
Does the first question seem a bit much of a 'gotcha'? I was slightly annoyed I got it wrong despite…
How can I get permission to edit this page, please?
I don't understand? A morphism is just an abstract element of a category. Its behaviour is completel…
I was thinking of starting here, then splitting into lenses once the structure is more certain. Do y…
I'd like to add some pictures to this page at some point, but due to current circumstances I can't f…
Joke stolen shamelessly from the latest post on slatestarcodex.com
Thanks Nate Soares!
Yeah the page is still under development and I'm planning to add more negative …
The urls are displaying as:
https://arbital.com/learn/?path=$bayes_rule_details,$bayes_update_detail…
This is still very much a work in progress. Anyone is welcome to submit more info or edit. I'll add …
Yeah, I think keeping it as it is now is probably the best way of following the "one idea per page" …
Patrick LaVictoire I think that's a fair assumption for the moment. Later as Arbital grows the requi…
Eric Bruylant Thank you very much! just to be clear, are you talking about the 'clickbait', the intr…
Eric Bruylant That would be great, thanks! I've sent you an email.
Eric Bruylant Whether Product (mathematics) is appropriate really depends if you're asking a categor…
Patrick Stevens I agree completely. Along with some other pictures. However, due tomy current circum…
Patrick Stevens Yeah I've been wondering about the convention of things like this. I've been calling…
Jaime Sevilla Molina
I've submitted an edit to the page on morphisms and wrote an intuitive guide t…
Eric Rogstad But... but... poset office was a *pun*, not a typo.
Eric Rogstad Elmo comes to visit. Does that seem fine you think?
Eric Rogstad The post has been updated with an isomorphic version of what you suggested. Thanks!
wiki
- Decimal notation
The winning architecture for numerals
- Derivative
How things change
- Expected value
Trying to assign value to an uncertain state? The weighted average of outcomes is probably the tool you need.
- Factorial
The number of ways you can order things. (Alternately subtitled: Is that exclamation point a factorial, or are you just excited to see me?)
- Integer
An **integer** is a Number that can be represented as either a Natural number or its [-additive\_inv…
- Inverse function
The inverse of a function returns an input of the original function when fed the original's corresponding output.
- Pi
Pi, usually written $π$, is a number equal to the ratio of a circle's [-circumference] to its [-diam…
- Real number
A **real number** is any number that can be used to represent a physical quantity.
Intuitively, rea…
- Asymptotic Notation
Asymptotic notation seeks to capture the behavior of functions as its input(s) become extreme. It is most widely used in Computer Science and Numerical Approximation.
wiki
- A googol
A pretty small large number.
- A googolplex
A moderately large number, as large numbers go.
- Abelian group
A group where the operation commutes. Named after Niels Henrik Abel.
- Abstract algebra
The study of groups, fields, vector spaces, arithmetics, algebras, and more.
- Algebraic structure
Roughly speaking, an algebraic structure is a set $X$, known as the underlying set, paired with a co…
- Arity (of a function)
The arity of a function is the number of parameters that it takes. For example, the function $f(a, b…
- Associative operation
An **associative operation** $\bullet : X \times X \to X$ is a binary operation such that for all $x…
- Associativity vs commutativity
Associativity and commutativity are often confused, because they are both constraints on how a funct…
- Associativity: Examples
Yes: [Addition], [multiplication], string concatenation. No: [subtraction], [division], a Function …
- Associativity: Intuition
Associative functions can be interpreted as families of functions that reduce lists down to a singl…
- Bag
In mathematics, a "bag" is an unordered list. A bag differs from a set in that it can contain the sa…
- Bayes' rule: Definition
Bayes' rule is the mathematics of probability theory governing how to update your beliefs in the lig…
- Bayes' rule: Probability form
The original formulation of Bayes' rule.
- Binary function
A binary function $f$ is a function of two inputs (i.e., a function with arity 2). For example, $+,$…
- Bit
The term "bit" refers to different concepts in different fields. The common theme across all the us…
- Bit (abstract)
An abstract bit is an element of the set $\mathbb B$, which has two elements. An abstract bit is to …
- Bit (of data)
A bit of data is the amount of data required to single out one message from a set of two. Equivalen…
- Bit (of data): Examples
In the game "20 questions", one player (the "leader") thinks of a concept, and the other players ask…
- Blue oysters
A probability problem about blue oysters.
- Cartesian product
The Cartesian product of two sets $A$ and $B,$ denoted $A \times B,$ is the set of all [ordered\_pai…
- Ceiling
The ceiling of a real number $x,$ denoted $\lceil x \rceil$ or sometimes $\operatorname{ceil}(x),$ i…
- Closure
A set $S$ is _closed_ under an operation $f$ if, whenever $f$ is fed elements of $S$, it produces an…
- Codomain (of a function)
The codomain $\operatorname{cod}(f)$ of a function $f : X \to Y$ is $Y$, the set of possible outputs…
- Codomain vs image
It is useful to distinguish codomain from image both (a) when the type of thing that the function pr…
- Communication: magician example
Imagine that you and I are both magicians, performing a trick where I think of a card from a deck of…
- Commutative operation
A commutative function $f$ is a function that takes multiple inputs from a set $X$ and produces an o…
- Commutativity: Examples
Yes: addition, multiplication, maximum, minimum, rock-paper-scissors. No: subtraction, division, st…
- Commutativity: Intuition
We can think of commutativity either as an artifact of notation, or as a symmetry in the output of a…
- Compressing multiple messages
How many bits of data does it take to encode an $n$-message? Naively, the answer is $\lceil \log_2(n…
- Conditional probability: Refresher
Is P(yellow | banana) the probability that a banana is yellow, or the probability that a yellow thing is a banana?
- Correspondence visualizations for different interpretations of "probability"
Let's say you have a model which says a particular coin is 70% likely to be heads. How should we as…
- Corrigibility
"I can't let you do that, Dave."
- Data capacity
The data capacity of an object is defined to be the Logarithm of the number of different distinguish…
- Decit
Decimal digit
- Dependent messages can be encoded cheaply
Say you want to transmit a 2-message, a 4-message, and a 256-message to somebody. For example, you m…
- Digit wheel
A mechanical device for storing a number from 0 to 9.
The domain $\operatorname{dom}(f)$ of a function $f : X \to Y$ is $X$, the set of valid inputs for t…
- Emulating digits
In general, given enough $n$-digits, you can emulate an $m$-digit, for any $m, n \in$ $\mathbb N$. I…
- Encoding trits with GalCom bits
There are $\log_2(3) \approx 1.585$ bits to a Trit. Why is it that particular value? Consider the Ga…
- Example problem
Tag for pages that provide an example problem referenced by a number of other pages.
The summary of…
- Exchange rates between digits
In terms of data storage, if a coin is worth $1, a digit wheel is worth more than $3.32, but less than $3.33. Why?
- Fractional bits
It takes $\log_2(8) = 3$ bits of data to carry one message from a set of 8 possible messages. Simila…
- Fractional bits: Digit usage interpretation
It is 316, not 500, that requires about two and a half digits to write down. 500 requires nearly 2.7…
- Fractional bits: Expected cost interpretation
In the GalCom thought experiment, you regularly have to send large volumes of information through de…
- Fractional digits
When $b$ and $x$ are integers, $\log_b(x)$ has a few good interpretations. It's roughly the length o…
- Frequency diagram
Visualizing Bayes' rule by manipulating frequencies in large populations
- Frequency diagrams: A first look at Bayes
The most straightforward visualization of Bayes' rule.
- Function
Intuitively, a function $f$ is a procedure (or machine) that takes an input and performs some opera…
- Function: Physical metaphor
Many functions can be visualized as physical mechanisms of wheels and gears, that take their inputs …
- GalCom
In the GalCom thought experiment, you live in the future, and make your money by living in the Dene…
- GalCom: Rules
1. It costs 1 galcoin per bit to reserve on-peak bits in advance. (Galcoins are very expensive.)
2. …
- Generalized associative law
Given an associative operator $\cdot$ and a list $[a, b, c, \ldots]$ of parameters, all ways of red…
- Graham's number
A fairly large number, as numbers go.
- Group
The algebraic structure that captures symmetry, relationships between transformations, and part of what multiplication and addition have in common.
- Group theory
What kinds of symmetry can an object have?
- How many bits to a trit?
$\log_2(3) \approx 1.585.$ This can be interpreted a few different ways:
1. If you multiply the nu…
- Image (of a function)
The image $\operatorname{im}(f)$ of a function $f : X \to Y$ is the set of all possible outputs of $…
- Information
Information is a measure of how much a message grants an observer the ability to predict the world.…
- Information theory
The study (and quantificaiton) of information, and its communication and storage.
- Interpretations of "probability"
What does it *mean* to say that a fair coin has a 50% probability of coming up heads?
- Intradependent encoding
An encoding $E(m)$ of a message $m$ is intradependent if the fact that $E(m)$ encodes $m$ can be de…
- Intradependent encodings can be compressed
Given an encoding scheme $E$ which gives an Intradependent encoding of a message $m,$ we can in prin…
- Introductory guide to logarithms
Welcome to the Arbital introduction to logarithms! In modern education, logarithms are often mention…
- Life in logspace
The log lattice hints at the reason that engineers, scientists, and AI researchers find logarithms s…
- Likelihood
"Likelihood", when speaking of Bayesian reasoning, denotes *the probability of an observation, sup…
- Likelihood function
Let's say you have a piece of evidence $e$ and a set of hypotheses $\mathcal H.$ Each $H_i \in \math…
- Likelihood notation
The likelihood of a piece of evidence $e$ according to a hypothesis $H,$ known as "the likelihood of…
- Likelihood ratio
Given a piece of evidence $e$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them…
- Linear algebra
The study of [linear\_transformation linear transformations] and vector spaces.
- List
A list is an ordered collection of objects, such as `[0, 1, 2, 3]` or `["red", "blue", 0, "shoe"]`. …
- Log as generalized length
To estimate the log (base 10) of a number, count how many digits it has.
- Log as the change in the cost of communicating
When interpreting logarithms as a generalization of the notion of "length" and as digit exchange rat…
- Log base infinity
There is no log base infinity, but if there were, it would send everything to zero
- Logarithm
The logarithm base $b$ of a number $n,$ written $\log_b(n),$ is the answer to the question "how man…
- Logarithm base 1
There is no log base 1.
- Logarithm tutorial overview
The logarithm tutorial covers the following six subjects:
1. What are logarithms?
2. Logarithms as…
- Logarithm: Examples
$\log_{10}(100)=2.$ $\log_2(4)=2.$ $\log_2(3)\approx 1.58.$ (TODO)
- Logarithm: Exercises
Without using a calculator: What is $\log_{10}(4321)$? What integer is it larger than, what integer …
- Logarithmic identities
- [ Inversion of exponentials]: $b^{\log_b(n)} = \log_b(b^n) = n.$
- [ Log of 1 is 0]: $\log_b(1) …
- Logarithms invert exponentials
The function $\log_b(\cdot)$ inverts the function $b^{(\cdot)}.$ In other words, $\log_b(n) = x$ imp…
- Monoid
A monoid $M$ is a pair $(X, \diamond)$ where $X$ is a [set\_theory\_set set] and $\diamond$ is an [a…
- Nate's ruminations
These posts are a mirror of posts on the blog [MindingOurWay.com](mindingourway.com) which pertain t…
- Needs lenses
This page has only a technical introduction. If you're able to, please help by adding an intuitive explanation!
- Non-standard terminology
A tag for terminology that is Arbital-specific, Arbital-originated, or just not very common outside …
- Odds form to probability form
The odds form of Bayes' rule works for any two hypotheses $H_i$ and $H_j,$ and looks like this:
$$\…
- Odds: Introduction
What's the difference between probabilities and odds? Why is a 20% probability of success equivalent to 1 : 4 odds favoring success?
- Odds: Refresher
A quick review of the notations and mathematical behaviors for odds (e.g. odds of 1 : 2 for drawing a red ball vs. green ball from a barrel).
- Operator
An operation $f$ on a set $S$ is a function that takes some values from $S$ and produces a new value…
- Order of a group
The order $|G|$ of a group $G$ is the size of its underlying set. For example, if $G=(X,\bullet)$ an…
- Ordinary claims require ordinary evidence
Extraordinary claims require extraordinary evidence, but ordinary claims *don't*.
- Probability distribution: Motivated definition
People keep writing things like P(sick)=0.3. What does this mean, on a technical level?
- Probability interpretations: Examples
Consider evaluating, in June of 2016, the question: "What is the probability of Hillary Clinton wi…
- Proof of Bayes' rule: Probability form
Let $\mathbf H$ be a [random\_variable variable] in $\mathbb P$ for the true hypothesis, and let $H_…
- Properties of the logarithm
- $\log_b(x \cdot y) = \log_b(x) + \log_b(y)$ for any $b$, this is the defining characteristic of …
- Range (of a function)
The "range" of a function is an ambiguous term that is generally used to refer to either the functio…
- Replacing Guilt
In my experience, many people are motivated primarily by either guilt, shame, or some combination of…
- Report likelihoods not p-values: FAQ
This page answers frequently asked questions about the Report likelihoods, not p-values proposal for…
- Report likelihoods, not p-values
If scientists reported likelihood functions instead of p-values, this could help science avoid p-ha…
- Ring
A ring is a kind of Algebraic structure which we obtain by considering groups as being "things with…
- Set
An unordered collection of distinct objects.
- Set builder notation
$\{ 2n \mid n \in \mathbb N \}$ denotes the set of all even numbers, using set builder notation. Set…
- Shannon
The shannon (Sh) is a unit of Information. One shannon is the difference in [info\_entropy entropy] …
- Shift towards the hypothesis of least surprise
When you see new evidence, ask: which hypothesis is *least surprised?*
- Size of a set
- Sock-dresser search
There's a 4/5 chance your socks are in one of your dresser's 8 drawers. You check 6 drawers at random. What's the probability they'll be in the next drawer you check?
- Sparking widgets
10% of widgets are bad and 90% are good. 4% of good widgets emit sparks, and 12% of bad widgets emit…
- String (of text)
A string (of text) is a series of letters (often denoted by quote marks), such as `"abcd"` or `"hell…
- Subspace
A subspace $U=(F_U, V_U)$ of a Vector space $W=(F_W, V_W)$ is a vector space where $F_U = F_W$ and $…
- Sum of vector spaces
The sum of two vector spaces $U$ and $W,$ written $U + W,$ is a vector space where the set of vector…
- The End (of the basic log tutorial)
That concludes our introductory tutorial on logarithms! You have made it to the end.
Throughout thi…
- The Stamp Collector
Once upon a time, a group of naïve philosophers found a robot that collected trinkets. Well, more sp…
- The characteristic of the logarithm
Any time you find an output that adds whenever the input multiplies, you're probably looking at a (…
- The log lattice
Log as the change in the cost of communicating and other pages give physical interpretations of what…
- There is only one logarithm
All logarithm functions are the same, up to a multiplicative constant.
- Thought experiment
Meta-tag for thought experiments.
- Trit
Trinary digit
- Underlying set
What do a Group, a Partially ordered set, and a [ topological space] have in common? Each is a Set …
- Vector space
A vector space is a field $F$ paired with a Group $V$ and a function $\cdot : F \times V \to V$ (cal…
- What is a logarithm?
Logarithms are a group of functions that take a number as input and produce another number. There i…
- Why is log like length?
If a number $x$ is $n$ digits long (in Decimal notation), then its logarithm (base 10) is between $n…
- Why is the decimal expansion of log2(3) infinite?
Because 2 and 3 are relatively prime.
- You're allowed to fight for something
The first sort of guilt I want to address is the listless guilt, that vague feeling one gets after p…
- concat (function)
The string concatenation function `concat` puts two strings together, i.e., `concat("one","two")="on…
- n-digit
An $n$-digit is a physical object that can be stably placed into any of $n$ distinguishable states. …
- n-message
A message singling out one thing from a set of $n$ is sometimes called an $n$-message. For example,…
(5) was intended to assume that $n \in \mathbb R^{\ge 1},$ or possibly $\in \mathbb R^{\ge 0}$ if yo…
:, not '
Added a verbalization of the image. Does that help?
Examples?
FYI, it's either "X is composed of Y" or "X comprises Y" (according to the dictionary definition, at…
Fixed, thanks.
Fixed, thanks.
Fixed, thanks.
Fixed. (Would be nice to have a way to resolve these comments.)
Fixed.
How are these changes? (starting at prop 5, through the end)
How's this?
I mean, I wasn't going to say it myself, but that was my general impression :-p
(I'm also dubious o…
I recommend rethinking the magnet metaphor, on the grounds that it is physically wrong. If you have …
I suggest adding more negative examples. (It's hard to learn a concept from only positive examples.)
I suggest making it explicit that $P$ is a distribution over a (possibly infinite) set of variables …
I think we don't need *that* much handholding. You've got to let people figure out some of the conne…
I tweaked this to make it a bit clearer while also cleaning up the latex (I think it's standard to u…
If those are the only two options, then you've gone mad :-)
L(H|e) is defined to be P(e|H) (which, …
If you're going to start using probability density functions instead of just probability functions, …
Made a minor edit. If you want anything more, you'll need to be more specific.
No (and it won't, until someone starts writing good explanations of radicals). I think it's fine to …
No. edited for clarity, see if that helps.
Sorry :-p /p/3vp seemed too general a name for me back when I was first writing this, though I think…
Suggestion: Mark this thread as an "editor only" comment.
What's with the %% marks? Can all \[square bracket thingies\] be turned into percent-demarcated bloc…
Yep. Fixed, thanks.
Yes :-p
Yes.
would add "by highlighting text and using the menu that appears on the margin" (or other text that b…
wut
yep, nice catch.
{{lowercase title}} is for things like eBay and iPod, IIRC -- which arbital should support eventuall…
- Being Strategic: The Very Idea
Being strategic (here synonymous with "rationality" and "optimization") means systematically working…
- Noticing Preferences
- Processes of Rationality
An idiosyncratic organization of rationality materials.
- Trying Things
Optimization requires trying things and being wrong.
- Astronomical Altruism
Playing for the huge outcomes
wiki
- Active learning for opaque, powerful predictors
####
(An open theoretical question relevant to AI control.)
Suppose that I have a very powerful p…
- Children in a sidebar
I always forget to scroll all the way down. I shouldn't have to. Putting children in a sidebar allow…
- Collusion is a major concern
Breaking out of the Prisoner's Dilemma
- Human in the counterfactual loop
Consider an autonomous system which is buying or selling assets, operating heavy machinery, or makin…
- Learning representations
Many AI systems form internal representations of their current environment or of particular data. Pr…
- Reinforcement learning
Reinforcement learning is the process by which an agent learns what actions to take by maximizing a …
- Reward engineering
This post gestures at a handful of research questions with a loose thematic connection.
## The idea…
- Taking people seriously
wiki
- Gödel encoding and self-reference
The formalism that mathematicians use to talk about arithmetic turns out to be able to talk about itself.
- Peano Arithmetic
A system for proving theorems about arithmetic, which is strong enough to include self-reference.
- Quine
A computer program that prints (or does other computations to) its own source code, using indirect self-reference.
A donor lottery could come with an agreement (which can only be enforced by honor) to *actually* spe…
At this point, I think the evidence points away from there being any deeply useful form of optimalit…
I suggest we can assume that almost everyone in Math 3 is familiar with either calculus concepts or …
I think nitpicks are a problem on LW, not because they clog the comments, but because the expectatio…
I'd agree more strongly if it had a couple of fixes for obvious problems, so I proposed some.
Probl…
It wasn't obvious to me in the UI whether, if I wanted to give feedback and then continue on to the …
Lots of null Arbital links that don't even connect to page stubs...
Maybe a useful thing to add: when we say things like "if X goes wrong, I expect your AI to become a …
Please clarify this claim, since there's an enormous difference between recruitment and outreach, be…
Possible inferential gap given just the pages I saw on my path to this one: the notion of "causally…
This is one of two ways I know of proving Löb's theorem, and I find them both illuminating. (The oth…
Two ways impeachment could happen:
- Trump becomes an albatross on the GOP, to the degree that the…
wiki
- Addition of rational numbers (Math 0)
The simplest operation on rational numbers is addition.
- Addition of rational numbers exercises
Test and cement your understanding of how we add rational numbers!
- Algebraic field
A field is a structure with addition, multiplication and division.
- Alternating group
The alternating group is the only normal subgroup of the symmetric group (on five or more generators).
- Alternating group is generated by its three-cycles
A useful result which lets us prove things about the alternating group more easily.
- Arithmetic of rational numbers (Math 0)
How do we combine rational numbers together?
- Bijective function
A bijective function is a function with an inverse.
- Bézout's theorem
Bézout's theorem is an important link between highest common factors and the integer solutions of a certain equation.
- Cantor-Schröder-Bernstein theorem
This theorem tells us that comparing sizes of sets makes sense: if one set is smaller than another, and the other is smaller than the one, then they are the same size.
- Category of finite sets
The category of finite sets is exactly what it claims to be. It's a useful training ground for some of the ideas of category theory.
- Cauchy's theorem on subgroup existence
Cauchy's theorem is a useful condition for the existence of cyclic subgroups of finite groups.
- Cauchy's theorem on subgroup existence: intuitive version
Cauchy's Theorem states that if $G$ is a finite [-group], and $p$ is a prime dividing the order of $…
- Cayley's Theorem on symmetric groups
The "fundamental theorem of symmetry", forging the connection between symmetry and group theory.
- Conjugacy class
In a group, the elements can be partitioned naturally into certain classes.
- Conjugacy class is cycle type in symmetric group
There is a neat characterisation of the conjugacy classes in the symmetric group on a finite set.
- Conjugacy classes of the alternating group on five elements
$A_5$ has easily-characterised conjugacy classes, based on a rather surprising theorem about when conjugacy classes in the symmetric group split.
- Conjugacy classes of the alternating group on five elements: Simpler proof
A listing of the conjugacy classes of the alternating group on five letters, without using heavy theory.
- Conjugacy classes of the symmetric group on five elements
The symmetric group on five elements is a group of just the right size to make a good example of a table of conjugacy classes.
- Cycle notation in symmetric groups
Cycle notation is a convenient way to represent the elements of a symmetric group.
- Cycle type of a permutation
The cycle type is an invariant of a permutation in the symmetric group.
- Cyclic group
Cyclic groups form one of the most simple classes of groups.
- Dihedral group
The dihedral groups are natural examples of groups, arising from the symmetries of regular polygons.
- Dihedral groups are non-abelian
The group of symmetries of the triangle and all larger regular polyhedra are not abelian.
- Disjoint cycle notation is unique
Disjoint cycle notation provides a canonical way to express elements of the symmetric group.
- Disjoint cycles commute in symmetric groups
In cycle notation, if two cycles are disjoint, then they commute.
- Disjoint union of sets
One of the most basic ways we have of joining two sets together.
- Division of rational numbers (Math 0)
"Division" is the idea of "dividing something up among some people so that we can give equal amounts to each person".
- Empty set
The empty set does what it says on the tin: it is the set which is empty.
- Empty set
The empty set, $\emptyset$, is the set with no elements. For every object $x$, $x$ is not in $\empt…
- Equaliser (category theory)
In Category theory, an *equaliser* of a pair of arrows $f, g: A \to B$ is an object $E$ and a univer…
- Euclid's Lemma on prime numbers
A very basic but vitally important property of the prime numbers is that they "can't be split between factors": if a prime divides a product then it must divide one of the individual factors.
- Euclidean domains are principal ideal domains
A Euclidean domain is one where we may somehow perform the division algorithm; this gives us access to some of the nicest properties of the integers.
- Every group is a quotient of a free group
Given a group $G$, there is a Free group $F(X)$ on some set $X$, such that $G$ is isomorphic to some…
- Every member of a symmetric group on finitely many elements is a product of transpositions
This fact can often simplify arguments about permutations: if we can show that something holds for transpositions, and that it holds for products, then it holds for everything.
- Factorial
The *factorial* of a number $n$ is how we describe "how many different ways we can arrange $n$ obje…
- Field homomorphism is trivial or injective
Field homomorphisms preserve a *lot* of structure; they preserve so much structure that they are always either injective or totally boring.
- Field structure of rational numbers
In which we describe the field structure on the rationals.
- Finite set
A finite set is one which is not infinite. Some of these are the least complicated sets.
- Formal definition of the free group
Van der Waerden's trick lets us define the free groups in a slick but mostly incomprehensible way.
- Free group
The free group is "the purest way to make a group containing a given set".
- Free group universal property
The Free group may be defined by a Universal property, allowing Category theory to talk about free …
- Free groups are torsion-free
An easy way to determine that many groups are not free: free groups contain no non-identity elements of finite order.
- Freely reduced word
"Freely reduced" captures the idea of "no cancellation" in a free group.
- Fundamental Theorem of Arithmetic
The FTA tells us that natural numbers can be decomposed uniquely into prime factors; it is the basis of almost all number theory.
- Greatest common divisor
The greatest common divisor of two natural numbers is… the largest number which is a divisor of both. The clue is in the name, really.
- Greatest lower bound in a poset
The greatest lower bound is an abstraction of the idea of the greatest common divisor to a general poset.
- Group action induces homomorphism to the symmetric group
We can view group actions as "bundles of homomorphisms" which behave in a certain way.
- Group conjugate
Conjugation lets us perform permutations "from the point of view of" another permutation.
- Group coset
Given a subgroup $H$ of Group $G$, the *left cosets* of $H$ in $G$ are sets of the form $\{ gh : h \…
- Group homomorphism
A group homomorphism is a "function between groups" that "respects the group structure".
- Group isomorphism
"Isomorphism" is the proper notion of "sameness" or "equality" among groups.
- Group orbits partition
When a group acts on a set, the set falls naturally into distinct pieces, where the group action only permutes elements within any given piece, not between them.
- Group presentation
Presentations are a fairly compact way of expressing groups.
- Ideals are the same thing as kernels of ring homomorphisms
In ring theory, the notion of "[ideal\_ring\_theory ideal]" corresponds precisely with the notion o…
- Image of the identity under a group homomorphism is the identity
All group homomorphisms preserve the identity.
- In a principal ideal domain, "prime" and "irreducible" are the same
Principal ideal domains have a very useful property that we don't need to distinguish between the informal notion of "prime" (i.e. "irreducible") and the formal notion.
- Index two subgroup of group is normal
An easy (though not very widely applicable) criterion for a subgroup to be normal.
- Injective function
A Function $f: X \to Y$ is *injective* if it has the property that whenever $f(x) = f(y)$, it is the…
- Integral domain
An integral domain is a ring where the only way to express zero as a product is by having zero as one of the terms.
- Irreducible element (ring theory)
This is the appropriate abstraction of the concept of "prime number" to general rings.
- Kernel of group homomorphism
The kernel of a Group homomorphism $f: G \to H$ is the collection of all elements $g$ in $G$ such th…
- Kernel of ring homomorphism
The kernel of a ring homomorphism is the collection of things which that homomorphism sends to 0.
- Lagrange theorem on subgroup size
Lagrange's Theorem is an important restriction on the sizes of subgroups of a finite group.
- Lagrange theorem on subgroup size: Intuitive version
Lagrange's theorem strongly restricts the size a subgroup of a group can be.
- Left cosets are all in bijection
The left cosets of a subgroup in a parent group are all the same size.
- Left cosets partition the parent group
In a group, every element has a unique coset in which it lies, allowing us to compress some of the information about the group.
- Multiplication of rational numbers (Math 0)
"Multiplication" is the idea of "now do the same as you just did, but instead of doing it to one apple, do it to some other number".
- Normal subgroup
Normal subgroups are subgroups which are in some sense "the same from all points of view".
- Object identity via interactions
If we think of objects as opaque "black boxes", how can we tell whether two objects are different? By looking at how they interact with other objects!
- Orbit-stabiliser theorem
The Orbit-Stabiliser theorem tells us a lot about how a group acts on a given element.
- Order of a group element
Given an element $g$ of group $(G, +)$ (which henceforth we abbreviate simply as $G$), the order of …
- Order of rational operations (Math 0)
Our shorthand for all the operations on rationals is very useful, but full of brackets; this is how to get rid of some of the brackets.
- Ordering of rational numbers (Math 0)
How do we know if one lot of apples is "more apples" than another lot?
- Partial function
A partial function is one which "might not be defined everywhere one might expect it to be".
- Pi is irrational
The number pi is famously not rational, in spite of joking attempts at legislation to fix its value at 3 or 22/7.
- Prime element of a ring
Despite the name, "prime" in ring theory refers not to elements which are "multiplicatively irreducible" but to those such that if they divide a product then they divide some term of the product.
- Prime number
The prime numbers are the "building blocks" of the counting numbers.
- Prime order groups are cyclic
This is the first step on the road to classifying the finite groups.
- Principal ideal domain
A principal ideal domain is a kind of ring, in which all ideals have a certain nice form.
- Product is unique up to isomorphism
If something satisfies the universal property of the product, then it is uniquely specified by that property, up to isomorphism.
- Project proposal: Intro to the Universal Property
Proposal for one of the first Arbital Projects.
- Proof of Rice's theorem
A standalone proof of Rice's theorem, including one surprising lemma.
- Quotient by subgroup is well defined if and only if subgroup is normal
Let $G$ be a Group and $N$ a Normal subgroup of $G$.
Then we may define the *quotient group* $G/N$ t…
- Rational arithmetic all works together
The various operations of arithmetic all play nicely together in a certain specific way.
- Rational number
The rational numbers are "fractions".
- Rational numbers: Intro (Math 0)
The rational numbers are "fractions". While the natural numbers measure the answer to the question …
- Real number (as Cauchy sequence)
There are several ways to construct real numbers; this is the most natural way to use them in computations.
- Rice's Theorem
Rice's Theorem tells us that if we want to determine pretty much anything about the behaviour of an arbitrary computer program, we can't in general do better than just running it.
- Set product
A fundamental way of combining sets is to take their product, making a set that contains all tuples of elements from the originals.
- Sign homomorphism (from the symmetric group)
The sign homomorphism is how we extract the alternating group from the symmetric group.
- Simple group
The simple groups form the "building blocks" of group theory, analogously to the prime numbers in number theory.
- Splitting conjugacy classes in alternating group
The conjugacy classes in the alternating group are usually the same as those in the symmetric group; there is a surprisingly simple condition for when this does not hold.
- Stabiliser (of a group action)
If a group acts on a set, it is useful to consider which elements of the group don't move a certain element of the set.
- Stabiliser is a subgroup
Given a group acting on a set, each element of the set induces a subgroup of the group.
- Subgroup is normal if and only if it is a union of conjugacy classes
A useful way to think about normal subgroups, which meshes with their "closed under conjugation" interpretation.
- Subgroup is normal if and only if it is the kernel of a homomorphism
The "correct way" to think about normal subgroups is as kernels of homomorphisms.
- Subtraction of rational numbers (Math 0)
In which we meet anti-apples.
- Surjective function
A surjective function is one which "hits everything in the codomain".
- Symmetric group
The symmetric groups form the fundamental link between group theory and the notion of symmetry.
- The alternating group on five elements is simple
The smallest (nontrivial) simple group is the alternating group on five elements.
- The alternating group on five elements is simple: Simpler proof
A proof which avoids some of the heavy machinery of the main proof.
- The alternating groups on more than four letters are simple
The alternating groups are the most accessible examples of simple groups, and this fact also tells us that the symmetric groups are "complicated" in some sense.
- The collection of even-signed permutations is a group
This proves the well-definedness of one particular definition of the alternating group.
- The composition of two group homomorphisms is a homomorphism
The collection of group homomorphisms is closed under composition.
- The empty set is the only set which satisfies the universal property of the empty set
This theorem tells us that the universal property provides a sensible way to define the empty set uniquely.
- The image of a group under a homomorphism is a subgroup of the codomain
Group homomorphisms take groups to groups, but it is additionally guaranteed that the elements they hit form a group.
- The rationals form a field
The set $\mathbb{Q}$ of rational numbers is a field.
# Proof
$\mathbb{Q}$ is a (commutative) ring …
- The reals (constructed as Dedekind cuts) form a field
The reals are an archetypal example of a field, but if we are to construct them from simpler objects, we need to show that our construction does indeed have the right properties.
- The reals (constructed as classes of Cauchy sequences of rationals) form a field
The reals are an archetypal example of a field, but if we are to construct them from simpler objects, we need to show that our construction does indeed have the right properties.
- The set of rational numbers is countable
Although there are "lots and lots" of rational numbers, there are still only countably many of them.
- The sign of a permutation is well-defined
This result is what allows the alternating group to exist.
- Transcendental number
A transcendental number is one which is not the root of any integer-coefficient polynomial.
- Transposition (as an element of a symmetric group)
A transposition is the simplest kind of permutation: it swaps two elements.
- Uncountability (Math 3)
Formal definition of uncountability, and foundational considerations.
- Under a group homomorphism, the image of the inverse is the inverse of the image
The operations of "taking inverses" and "applying a group homomorphism" commute: it does not matter in which order we do them.
- Unique factorisation domain
This is the correct way to abstract from the integers the fact that every integer can be written uniquely as a product of prime numbers.
- Unit (ring theory)
A unit in a ring is just an element with a multiplicative inverse.
- Universal property
A universal property is a way of defining an object based purely on how it interacts with other objects, rather than by any internal property of the object itself.
- Universal property of joins and meets in a poset
Partially ordered sets are a good setting to illustrate some of the concepts of category theory; joins and meets are instances of category-theoretic ideas.
- Universal property of the disjoint union
Just as the empty set may be described by a universal property, so too may the disjoint union of sets.
- Universal property of the empty set
The empty set can be characterised by how it interacts with other sets, rather than by any explicit property of the empty set itself.
- Universal property of the product
The product can be defined in a very general way, applicable to the natural numbers, to sets, to algebraic structures, and so on.
- Up to isomorphism
A phrase mathematicians use when saying "we only care about the structure of an object, not about specific implementation details of the object".
- Well-defined
A mathematical object is "well-defined" if we have given it a completely unambiguous definition.
"identity" is probably not a sufficiently specific link; I'd go for math_identity, probably.
A question about the requisites for this page: should The alternating group on five elements is simp…
A summary of the relevant cardinal arithmetic, by the way (in the presence of choice): $$\aleph_{\al…
Are you otherwise broadly Math 3? It would be good to have a guinea pig for group theory.
Have I gone mad, or do you mean "L(H|e) is simply the probability of H given that the the actual dat…
I don't think this is what you mean, is it?
I feel like symmetric_group should be a requisite for this page. However, this page is linked in the…
I have a question about general Arbital practice here. A mathematician will probably already know wh…
I think the answer is no. Indeed, there are uncountably many $S$, but only countably many machines w…
I think the halting problem probably should have its own page, rather than being linked to the umbre…
I think this actually belongs in the Multiplication article, but you're quite right that I've not be…
I think this probably wants a diagram of the two graphs, being differently laid out in the plane but…
I took the plunge and put it on its own page.
I've edited something about that into the text. Basically I think it's to do with the symmetry of th…
It sounds like you didn't already know what the free group is; in that case (and even if you did alr…
Looks good to me!
None that I'm aware of, but I've found it convenient to know when I was doing exercises in a first c…
Request for comment: is the definition of "cycle" something that should be on its own page? They're …
Simply that I didn't know the name :) I'll edit it in.
Something I learnt from Mietek Bak is that Löb's Theorem is kind of more subtle than this. In provab…
Surely they are equivalent. Given a Rice-deciding oracle, we can ask the oracle, "Does the partial f…
Thanks: quite correct.
The non-existence of a total order on $\mathbb{C}$ is fun and interesting, I think, and also not ver…
This is definitely a page which admits two lenses: the "easy" proof and the "theory-heavy" proof. Wh…
This is not universally agreed-upon, but I use "$A$ decides whether or not $B$ holds" to mean "$A$ o…
This page doesn't disambiguate between "left inverse" and "inverse". Strictly an "inverse" is a two-…
To the original author: xkcd images are CC BY-NC (2.5), and as such require attribution.
You're quite right to flag this up; I was being sloppy. There are three main ways to construct the f…
You're right; I was sloppy. I'll fix it, thanks.
Ilia Zaichuk Thanks for the edit! I made a couple of linguistic changes, and made the "uniqueness of…
[@5hc]: I've made the appropriate changes to the markup to make text display in MathJax (which is th…
wiki
- A possible stance for AI control research
I think that AI control research should focus on finding [scalable](https://arbital.com/pages/492374…
- AI control on the cheap
Ideally, we will build aligned AI systems without sacrificing any efficiency.
I think this is a rea…
- Abstract approval-direction
Consider the following design for an agent, which I first described [here](https://arbital.com/p/1t7…
- Act based agents
I’ve recently discussed three kinds of learning systems:
- [Approval-directed agents](https://arbit…
- Active learning for opaque, powerful predictors
(An open theoretical question relevant to AI control.)
Suppose that I have a very powerful predicti…
- Adversarial collaboration
Suppose that I have hired a group of employees who are much smarter than I am. For some tasks it’s…
- Advisor games
Machine learning algorithms often learn models or policies that are inscrutable to humans. We belie…
- Against mimicry
One simple and apparently safe AI system is a “copycat:” an agent that predicts what its user woul…
- Ambitious vs. narrow value learning
Suppose I’m trying to build an AI system that “learns what I want” and helps me get it. I think tha…
- Apprenticeship learning and mimicry
This post compares my [recent proposal](https://arbital.com/p/1vp/mimicry_meeting_halfway) with [Ab…
- Approval directed agents
Research in AI is steadily progressing towards more flexible, powerful, and autonomous goal-directe…
- Approval-based agents
An alternative to goal-directed behavior
- Approval-directed bootstrapping
Approval-directed behavior works best when the overseer is very smart. Where can we find a smart o…
- Automated assistants
In my [last post](https://arbital.com/p/1th?title=implementing-our-considered-judgment), I describ…
- Challenges for safe AI from RL
In this post, I’ll describe and discuss a few big problems for the proposal from [my last post](htt…
- Concrete approval-directed agents
This post lays out my current concrete “[approval-directed agents](https://arbital.com/p/1t7)” propo…
- Counterfactual oversight vs. training data
I have written a lot recently about [counterfactual human oversight](https://arbital.com/p/1tj?titl…
- Delegating to a mixed crowd
###
Suppose I have ten programs, each a human-level agent. I suspect that at least one or two of…
- Efficient feedback
In some machine learning domains, such as image classification, we can produce a bunch of labelled t…
- Elaborations on apprenticeship learning
Apprenticeship learning (AL) is an intuitively appealing approach to AI control. In AL, a human expe…
- Handling adversarial errors
Even a very powerful learning system can’t do everything perfectly at first — it requires time to l…
- Handling errors with arguments
My [recent proposal](https://arbital.com/p/1v7?title=steps-towards-safe-ai-from-online-learning) f…
- How common is imitation?
How often do we train machine learning systems to imitate human behavior?
Some researchers explicit…
- Human arguments and AI control
### Explanation and AI control
Consider the definition:
> An action is good to the extent that I w…
- Human in counterfactual loop
Consider an autonomous system which is buying or selling assets, operating heavy machinery, or mak…
- Humans consulting HCH
Consider a human who has access to a question-answering machine. Suppose the machine answers questio…
- IRL and VOI
Consider the following straightforward algorithm based on inverse reinforcement learning:
- Given…
- Imitation and justification
Suppose that I am training an AI system to play Go. One approach is to have the AI observe human m…
- Implementing our considered judgment
Suppose I had a very powerful prediction algorithm. How might I use this algorithm to build a smar…
- Implicit consequentialism
Consider a machine that does exactly what its user [would tell it to do](https://arbital.com/p/1tj?t…
- In defense of maximization
I’ve been thinking about [AI systems that take actions their users would most approve of](https://…
- Indirect decision theory
In which I argue that understanding decision theory can be delegated to AI.
### Indirect normativit…
- Learn policies or goals?
I’ve [recently proposed](https://arbital.com/p/1t7/approval_directed_agents) training agents to ma…
- Learning and logic
In most machine learning tasks, the learner maximizes a concrete, empirical performance measure: i…
- Learning representations
Many AI systems form internal representations of their current environment or of particular data. Pr…
- Mimicry and meeting halfway
I’ve talked recently about two different model-free decision procedures:
- At each step, pick the …
- Modeling AI control with humans
I’ve been trying to build an aligned AI out of reward-maximizing modules. A successful scheme could …
- Of arguments and wagers
(In which I explore an unusual way of combining the two.)
Suppose that Alice and Bob disagree, and …
- Of simulations and inductive definitions
_(Warning: weird.)_
Consider a simple AI system, named A, that carries out a task by predicting wha…
- On heterogeneous objectives
Eliezer Yudkowsky [has said](https://www.facebook.com/yudkowsky/posts/10153748345169228):
> If you …
- Online guarantees and AI control
I’m interested in claims of the form: “If we had an AI that could do X well, then we could build a…
- Optimization and goals
If we want to write a program that _doesn’t_ pursue a goal, we can have two kinds of trouble:
1. We…
- Optimizing with comparisons
I could [elicit a user’s approval](https://arbital.com/p/1w5) of an action _a_ by having them supply…
- Paul Christiano's AI control blog
Speculations on the design of safe, efficient AI systems.
- Problem: safe AI from episodic RL
In [a previous post](https://arbital.com/p/1tv?title=the-steering-problem), I posed the steering pr…
- Reinforcement learning and linguistic convention
Existing machine learning techniques are most effective when we can provide concrete feedback — such…
- Research directions in AI control
What research would best advance our understanding of AI control?
I’ve been thinking about this qu…
- Reward engineering
This post gestures at a handful of research questions with a loose thematic connection.
### The id…
- Safe AI from question-answering
_(Warning: minimal new content. Just a clearer framing.)_
Suppose that I have a question-answering…
- Scalable AI Control
By AI control, I mean the problem of getting AI systems to do what we want them to do, to the best o…
- Scalable AI control
By AI control, I mean the problem of getting AI systems to do what we want them to do, to the best…
- Stable self-improvement as an AI safety problem
“Stable self-improvement” seems to be a primary focus of MIRI’s work. As I understand it, the proble…
- Steps towards safe AI from online learning
### Steps towards safe AI from online learning
Suppose that we have a good algorithm for episodic r…
- Synthesizing training data
[Counterfactual oversight](https://arbital.com/p/1tj?title=human-in-counterfactual-loop) requires th…
- Technical and social approaches to AI safety
I often divide solutions to the AI control problem into two parts: technical and social. I think a…
- The absentee billionaire
Once each day, Hugh wakes for 10 minutes. During these 10 minutes, he spends 10 million dollars. The…
- The easy goal inference problem is still hard
Goal inference and inverse reinforcement learning
------------------------------------------------…
- The state of the steering problem
The [steering problem](https://arbital.com/p/1tv?title=the-steering-problem) asks: given some powe…
- The steering problem
Most AI research focuses on reproducing human abilities: to learn, infer, and reason; to perceive,…
- Unsupervised learning and AI control
Reinforcement learning systems optimize for an objective defined by external feedback — anything fro…
(This is hard without threaded conversations. Responding to the "agree/disagree" from Eliezer)
>The…
(Understandable to focus on explanation for now. Threaded replies to replies would also be great eve…
> If you want to demonstrate competence... you should first think in terms of exposing technically …
> >This is like one step of ten in the act-based approach, and so to the extent that we disagree it …
> Even so, while the outputs are still abstract and not-yet-computed, Alice doesn't have much of a p…
> on my view it seems extremely probable that, whatever we have in the way of AI algorithms short of…
A 5% change in mortality / 15% change in other endpoints would be surprisingly large to me. Does thi…
Act-based is a more general designation, that includes e.g. imitation learning (and value learning w…
As I see it, there are two cases that are meaningfully distinct:
(1) what we want is so simp…
Consider an AI system composed of many interacting subsystems, or a world containing many AI systems…
Eliezer [objects](https://arbital.com/p/2fr/?l=2fr#subpage-2h4) to this post's optimism about robust…
Eliezer seems to have, and this page seems to reflect, strong intuitions about "self-modification" b…
Eliezer, I find your position confusing.
Consider the first AI system that can reasonably predict y…
I agree that reflective degrees of freedom won't "fix themselves" automatically, and that this is a …
I agree. You can use the results of easier/earlier inferences to guide harder/later inferences, but …
I am pretty surprised by how confident the voters are!
Is "arbitrarily powerful" intended to includ…
I can imagine this concept becoming relevant one day. But it seems sufficiently improbable that it d…
I don't see indirect specifications as encountering these difficulties; all of the contenders so far…
I don't see why getting the satisfying assignment really matters. If your AI sometimes declines to a…
I don't think the existence of such a colony would directly mitigate AI risk, but it could help in t…
I don't think you've correctly diagnosed the disagreement yet (your strawman position is obviously c…
I expect you know my answer on this one.
I agree that if there is a *really* fast transition (e.g. …
I obviously disagree with "under intelligence explosion scenarios a Singleton seems like a quite pro…
I responded [here](https://arbital.com/p/1w4/?l=1w4#subpage-2hr).
> Some of this probably reflects …
I share the concern that people working on value alignment won't understand what has been done befor…
I think it's going to be hard to talk or think clearly about these problems (even at the level of se…
I think that using the uniform prior over observers constitutes a critical learning failure. Calling…
I think the [key question](https://medium.com/ai-control/the-informed-oversight-problem-1b51b4f66b35…
I was comparing act-based agents to what you are calling a genie. Both get objectives from humans an…
I was talking to Chelsea Finn about IRL a few weeks ago, and she said that they had encountered the …
I wouldn't call this "Christiano's hack." I appreciate the implicit praise that I can think up esote…
If the distinguishing characteristic of a genie is "primarily relying on the human ability to discer…
If there aren't side effects, it seems like the answer is probably yes, since vitamin D deficiency s…
In practice, Eliezer often invokes this concept in settings where there *isn't* yet an intelligent a…
In the long run automation will increase the share of income going to capital. I think theory is agn…
In the sudoku and first OWF example, the agent can justify their answer, and its easy to incentivize…
It seems critical to distinguish the cases where
1. We are hoping the AI generalizes the concept of…
It seems unlikely we'll ever build systems that "maximize X, but rule out some bad solutions with th…
It's easy to equivocate between "can be viewed as" and "is." Indeed, any rational agent "can be view…
It's worth pointing out that in our discussions of AI safety, the author (I assume Eliezer, hereafte…
Methodologically, I am trying to understand what approaches may or may not work and what the key dif…
My knee-jerk response to this problem (just as with mind crime and corrigibility) is to try to build…
My views about Eliezer's preferences may depend on the reason that I am running X, rather than merel…
Normally I think that you set the bar too high for yourself. In this case, I think that you would be…
Of course, the game is typically about costs and benefits. Saying "it is good to adopt the security …
Often complaints are with the particular problems which purportedly will require novel solutions or …
On the act-based model, the user would say something like "paint all the cars pink," and the AI woul…
On this definition, what is the difference between "communicating a goal concept" and "communicating…
One natural standard: it should be hard to distinguish an adequate model from the system-to-be-model…
Presumably the advantage of this approach---rather than simply learning to imitate the human burrito…
Re: "poking holes in things," what is an example of a proposal you would ask people to poke a hole i…
Re: simulating a hostile superintelligence:
I find this concern really unconcerning.
Some points:…
Regarding corporations:
I have seen very few arguments about superintelligence that rest on epistem…
Six months and several discussions later this still seems like a serious concern (Nick Bostrom seeme…
Sorry, I tried to be concrete about what we were discussing, but I will try harder:
Consider some p…
Superficially, there are two quite different concerns:
1. You optimize a system for X. You are unha…
The ZFC provability box is equivalent to a good SAT solver, up to a constant factor (and I don't see…
The obvious patch is for a sufficiently sophisticated system to have preferences over its own behavi…
There seems to be some equivocation here between two motivations for studying corrigibility.
As far…
These arguments seem weak to me.
- I think the basic issue is that you are not properly handling u…
This (and many of your concerns) seem basically sensible to me. But I tend to read them more broadly…
This is a more general pattern in theoretical research. When you first start to attack a hard proble…
This isn't the case in modern cryptography, except perhaps for the design of ciphers. It seems at be…
This seems like a good example to have at hand. I'm skeptical that it's much easier than what we rea…
This seems like a straw alternative. More realistically, we could imagine an agent which avoids pert…
This topic consistently frustrates me; the proposed typology is obviously incomplete, and I don't th…
To me, the most natural way to approach this is to take a probability distribution over "what it mea…
To the extent that humans can imagine these kinds of scenarios, it seems pretty futile to try to pre…
We can imagine two regimes of this problem: in the weak regime the AI may make a small number of err…
Yeah, thanks.
Your characterization of utility indifference doesn't seem quite right. More accurate would be: the …
wiki
- Colon-to notation
Find out what the notation "f : X -> Y" means that everyone keeps using.
- Group action
"Groups, as men, will be known by their actions."
- Group theory: Examples
What does thinking in terms of group theory actually look like? And what does it buy you?
- Group: Examples
Why would anyone have invented groups, anyway? What were the historically motivating examples, and what examples are important today?
- Group: Exercises
Test your understanding of the definition of a group with these exercises.
- In notation
There's a weird E-looking symbol called \in in LaTeX. What does it mean?
- Mapsto notation
There's an arrow called \mapsto in LaTeX. What does it mean?
- Representation theory
The study of how groups act on vector spaces.
wiki
- Practicing Brevity
Wanna save the world? Write better TLDRs. And spend more time respecting your reader's time.
wiki
- Regex's feedback on site design Feb 15th 2017
Wherein Regex is confused about the current state of things
wiki
- playpen subpage
playpen subpage clickbait
wiki
wiki
- Algebraic structure tree
When is a monoid a semilattice? What's the difference between a semigroup and a groupoid? Find out here!
- Reflexive relation
A binary relation over some set is **reflexive** when every element of that set is related to itself…
- CFS-spectrum disorders are caused by bacterial or viral infections
Chronic fatigue syndrome, fibromyalgia, Gulf War syndrome, and irritable bowel syndrome are often gr…
- Oropharyngeal cancer is a significant risk of HPV
HPV can cause oral and pharyngeal cancer. The incidence of HPV-associated oropharyngeal cancer is ab…
wiki
wiki
- Doppler Effect
Why do things make a higher pitch sound as they come toward me but lower as they go away?
- Externality
Positive and negative affects on third parties, and the considerations they introduce
- Human perception of sound
What is the mechanism by which vibrations around the human ear are translated into the sensation of sound?
- Pigovian tax
Taxation of negative externalities so that their producers have an incentive to cheaply reduce them
- Supply and Demand
How are prices typically formed in market systems?
wiki
- CFAR should explicitly focus on AI safety
The Center for Applied Rationality has historically had a "cause-neutral" mission but has recently r…
- Another another playpen child
May it be a light for you in dark places, when all other lights go out.
- Claim-tagging is worth trying more broadly
Source of claim: Improve comments by tagging claims by Benjamin Hoffman
- Comments are a high-quality, high-sensitivity measure of engagement with little in the way of viable substitutes.
Source of claim: Improve comments by tagging claims by Benjamin Hoffman
- Explicitly tagging the core claims of a post will make people substantially more likely to respond to these claims.
Source of claim: Improve comments by tagging claims by Benjamin Hoffman
- Groups trying to make intellectual progress need a central locus of discussion
For communities that are trying to make intellectual progress, it's important to have a single place…
- Headers demo
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore …
- Irrelevant nitpicks are an important problem in comment sections on sites such as LessWrong.
Source of claim: Improve comments by tagging claims by Benjamin Hoffman
- Location on the comments-links continuum is an important aspect of discourse design.
Source of claim: Improve comments by tagging claims by Benjamin Hoffman
- Meta tags which request an edit to the page
Tags that mean your page should be edited.
- The biggest source of human bias is the incentive to lead others to believe that you are better than you are
If evolution were optimizing creatures that were exactly like humans except that they were not socia…
- P-value
The probability of observing the sample data we have _actually_ observed; the sauce behind significance testing.
- Existence Proof of Logical Inductor
A procedure for constructing an algorithm that is a logical inductor relative to any given deductive set.
wiki
wiki
- Drinking on average 8 grams of alcohol a day will shorten your life expectancy
This is a very small amount equivalent to one UK unit. See https://en.wikipedia.org/wiki/Recommended…
wiki
- Intentional Communities wiki pages
A categorized, handpicked list of articles relevant to the Accelerator project
- Requisites for personal growth
A mashup of models
wiki
wiki
- Example: Dragon Pox
Kai might have Dragon Pox. Oy.
- Joint probability distribution
A probability distribution over the collection of joint configurations of all the variables you care about.
- Joint probability distribution: (Motivation) coherent probabilities
If you don't use joint probability distributions, none of your probabilities will make any sense. So, yeah, use joint probability distributions.
- Probability distribution (countable sample space)
A function assigning a probability to each point in the sample space.
- Sample space
The set of possible things that could happen in a part of the world that you are uncertain about.
- Sample spaces are too large
Sample spaces are often large, so it is hard to do probabilistic computations using a raw distribution over the sample space.
- Square visualization of probabilities on two events
$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
…
- Square visualization of probabilities on two events: (example) Diseasitis
But it *seems* like the patient with the black tongue depressor has diseasitis...
- Two independent events
What do [a pair of dice], [a pair of coins], and [a pair of people on opposite sides of the planet] all have in common?
- Two independent events: Square visualization
$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
…
- Uncountable sample spaces are way too large
We can't define probability distributions over uncountable sample spaces by just assigning numbers to each point in the sample space.
- Necessary conditions for expertise: the P-I-F-T method
A domain-general method to help you assess whether a person has the necessary conditions for expert in a given domain.
- Harry Potter at Ollivander's
A simple word problem on the application of Bayes rule.
wiki
- Eigenvalues and eigenvectors
We applied this linear transformation to one of its eigenvectors; you won't believe what happened next!
- AI and Machine Learning resources
Content that helped me with my AI/ML journey
wiki
- test page
blah blah blah
- test page 2
blah blah blah