Homeland Security is Using Text Profiling to Discriminate “Fact” from “Opinion”

Posted by Alex Zeka on September 26, 2006, 04:22 AM | #

I find all this highly suspicious. If I state that homosexuals have a lower time preference because of their inability to have children and hence lack of a stake in the future, is that a fact or an opinion? Is the Big Bang theory, which is doubted by many scientists, a fact or an opinion? For that matter, is this distinction between fact and opinion itself a fact or an opinion?

These are just a bunch of semi-autistic computer researchers who, not being able to themselves understand human discussion, are trying to reduce it what they can understand: computer algorithms. I’m not buying.

Posted by James Bowery on September 26, 2006, 01:11 PM | #

Don’t be confused by their failure to come up with an objective distinction (operational definition independent of human judgement) between “fact” vs “opinion”.  All they have done is ask some humans to use their judgement to classify some writings as “fact” and others as “opinion” and then used pretty standard data mining techniques to train a computer program to mimic that judgement against a much larger sample of texts.

The best the computer can do under these circumstances is no better than the selected human consensus can do.  Indeed, as I pointed out in an earlier comment on “word sense disambiguation” and its application to creation of coherent lexicons, the use of humans as the standard is precisely where these approaches are failing to realize the potential of computer algorithms.  There is a battle brewing within the philosophy of science over precisely this sort of standard and it is going to erupt throughout all of academia, the humanities as well as sciences.

The trigger of this eruption is the termination of the long hiatus—now nearly 50 years—of rational research into artificial intelligence.  I won’t go into all of the dimensions of the abominable history of artificial intelligence research, but suffice to say that with the resurgence of algorithmic information theory, things are being reformulated rapidly.

The bottom line is this:

Information and knowledge are inseparable.  If you can formulate information theory consilient with computer technology you have a rational basis for artificial intelligence.  Algorithmic information theory is that consilience and it has been in hibernation for decades.

The principle result of algorithmic information theory is that the shortest program that can output a text string represents the true information content of that text string.  It is Ockham’s Razor on steroids.

This doesn’t mean that a computer program can be written that will find that shortest program—indeed it has been proven that such a metaprogram cannot exist in the general sense.  But what it does mean is that we have an objective test of the relative truthfulness of two discriptive frameworks.  The one which results in the shortest description of the world—the one that is most coherent—most consilient—that “hangs together’ the best—is also the most truthful.  We can still have human judgement play a part of course—but that part is put to the emperical test of now rigorously defined epistemology.

The failure of “political correctness” as a conceptual framework is, like the failure of the canons of prior theocracies, due to their need to inject confusing political construct at the wrong level of discourse.  The correct level of discourse for political correctness, as with much theocratic nonsense, is as an instance of ethnic nepotism hijacking the moral machinery of competing ethnicities.  If placed at its proper place in the universe of discourse, the world becomes more comprehensible precisely because its description is simpler.

Posted by TNB Alerts on September 26, 2006, 11:24 PM | #

Here let me help them:

It is a fact that TNB is a major problem throughout the world today.  It is a fact that negroes are 13% of the US population but are responsible for over 50% of the crime.  Just the facts, man, just the facts.