Log in

No account? Create an account
Templar [userpic]

Idea: support vectors in AGI alignment problem.

May 24th, 2017 (01:54 pm)

(I need to mention that I am in no way AGI specialist, so maybe this will be quite naive...)

Usually when you hear/read about AGI-A problem, examples of what they are trying to achieve and what is considered for "training" are quite obvious and extreme - "killing people is bad, inventing disease cures is good" style. While this will be certainly useful for unit testing already aligned AGI, I don't think it will be so useful for training. Consider SVM type of algorithms that used for classification problems - what there called "support vectors" aren't extremes of their classes, quite contrary - they are most similar examples. You want your algorithm to learn subtle differences between very similar outcomes, not broad categories with 99% in "gray zone."

In a similar fashion you want to train AGI on controversial tasks, (in)famous "trolley problem" type and so on, specifying what your particular AI should consider the right choice.

Of course, this opens an entirely new can of worms, because humans themselves can't agree on what is the right thing in those problems, but this will allow more fine tuning of result AGI developers trying to achieve.