Philosophy of AI, from NI to AI, and Granular Computing:

---

• AAAI, AI topics.



---

January 6, 9, 2012

• Chapter 1, Textbook

What is artificial intelligence?

• Wikipedia, Artificial intelligence
• John McCarthy, WHAT IS ARTIFICIAL INTELLIGENCE?
• Jack Copeland, What is Artificial Intelligence?


What are the purposes and goals of artificial intelligence?
• Schank, R. C., What is AI, anyway? AI Magazine, 8(4), 59-65, 1986.
  "[AI's] primary goal is to build an intelligent machine. The second goal is to find out about the nature of intelligence."


• Herbert Simon, "AI can have two purposes. One is to use the power of computers to augment human thinking, just as we use motors to augment human or horse power. Robotics and expert systems are major branches of that. The other is to use a computer's artificial intelligence to understand how humans think. In a humanoid way. If you test your programs not merely by what they can accomplish, but how they accomplish it, they you're really doing cognitive science; you're using AI to understand the human mind.
  From, Stewart D (1994), Interview with Herbert Simon, OMNI Magazine.

---

January 11, 13, 16, 18, 2012

• Chapter 16, Textbook

From natural intelligence to artificial intelligence

• Wikipedia, Moravec's paradox.


• Robert C. Berwick, et al. The Human Intelligence Enterprise.


• National Academy of Engineering, Reverse-engineer the brain.


• Tom M. Mitchell, AI and the Impending Revolution in Brain Sciences, AAAI Presidential Address.


• Marvin Minsky, The Emotion Machine.


• O. G. Selfridge Pandemonium: A paradigm for learning. In D. V. Blake and A. M. Uttley, editors, Proceedings of the Symposium on Mechanisation of Thought Processes, pages 511-529, London, 1959.
• Wikipedia, Pandemonium architecture/


• Wikipedia, On Intelligence - description of the book by Jeff Hawkins.
  Official web site, OnIntelligence.com of On Intelligence.


• Wikipedia, A brief discussion on Pinker's book "How the Mind Works".
  Official web site, Pinker's website on How the Mind Works


• Jerry Fodor, The Trouble with Psychological Darwinism, London Review of Books Vol 20, No 2.
• Jerry Fodor, Introduction of the book: The Mind Doesn't Work That Way: The Scope and Limits of Computational Psychology Cambridge, MA: MIT Press, July 2000


• Stephen M. Downes, Evolutionary Psychology, Stanford Encyclopedia of Philosophy.
• Wikipedia, Evolutionary psychology.


• Steven Horst, The Computational Theory of Mind, Stanford Encyclopedia of Philosophy.
• Wikipedia, Computational theory of mind.


• Paul Thagard, Cognitive Science Stanford Encyclopedia of Philosophy.
• Wikipedia, Cognitive Science.

January 20, 23, 25, 27, 30, 2012

Concepts and Categorization:

• Margolis, E., Laurence, S. Concepts.


• Goldstone, R.L., Persten. A., Concepts and Categorization.


• Medin, D.L., Smith, E.E., Concepts and Concept Formation.

Machine learning (concept learning):

• Wikipedia, Information theory.


• J.R. Quinlan, Induction of decision trees, Machine Learning, Volume 1, Number 1, 81-106, 1986.
• Section 10.3, textbook.


• J CENDROWSKA, PRISM: An algorithm for inducing modular rules, lnL J. Man-Machine Studies (1987) 27, 349-370.


• Johannes Funkranz, Separate-and-Conquer Rule Learning, ARTIFICIAL INTELLIGENCE REVIEW Volume 13, Number 1, 3-54, 1999.


• Y.Y. Yao, Interpreting concept learning in cognitive informatics and granular computing, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 39(4): 855-866, 2009.

February 1, 3, 6, 8, 10, 13, 2012

A Triarchic Theory of Granular Computing

The needs and motivation for granular computing

• George A. Miller, The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, he Psychological Review, 1956, vol. 63, pp. 81-97.
  "In order to speak more precisely, therefore, we must recognize the importance of grouping or organizing the input sequence into units or chunks. Since the memory span is a fixed number of chunks, we can increase the number of bits of information that it contains simply by building larger and larger chunks, each chunk containing more information than before."
  "A man just beginning to learn radio-telegraphic code hears each dit and dah as a separate chunk. Soon he is able to organize these sounds into letters and then he can deal with the letters as chunks. Then the letters organize themselves as words, which are still larger chunks, and he begins to hear whole phrases. I do not mean that each step is a discrete process, or that plateaus must appear in his learning curve, for surely the levels of organization are achieved at different rates and overlap each other during the learning process. I am simply pointing to the obvious fact that the dits and dahs are organized by learning into patterns and that as these larger chunks emerge the amount of message that the operator can remember increases correspondingly."


• Jerry R. Hobbs, Granularity,
  "We look at the world under various grain sizes and abstract from it only those things that serve our present interests."
  "Our ability to conceptualize the world at different granularities and to switch among these granularities is fundamental to our intelligence and flexibility. It enables us to map the complexities of the world around us into simple theories that are computationally tractable to reason in. If we are to have a machine of even moderate intelligence, it must have a theory of granularity woven into the very foundation of its reasoning processes."


• Zadeh, L.A., Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy, Fuzzy Sets and Systems 90 (1997) 111-127.
  "There are three basic concepts that underlie human cognition: granulation, organization and causation. Informally, granulation involves decomposition of whole into parts; organization involves integration of parts into whole; and causation involves association of causes with effects."
  "Inspired by the ways in which humans granulate human concepts~ we can proceed to granulate conceptual structures in various fields of science. In a sense, this is what motivates computing with words. An intriguing possibility is to granulate the conceptual structure of mathematics. This would lead to what may be called granular mathematics. Eventually, granular mathematics may evolve into a distinct branch of mathematics having close links to the real world. A subset of granular mathematics and a superset of computing with words is granular computing."


• Yiyu Yao, Granular computing: basic issues and possible solutions, Proceedings of the 5th Joint Conference on Information Sciences, Volume I, pp. 186-189, 2000.



Multilevel Hierarchical Granular Structures

• Herbert A. Simon, THE ARCHITECTURE OF COMPLEXITY, Proceedings of the American Philosophical Society, Vol. 106, No. 6. (Dec. 12, 1962), pp. 467-482.
  "By a hierarchic system, or hierarchy, I mean a system that is composed of interrelated subsystems, each of the latter being, in turn, hierarchic in structure until we reach some lowest level of elementary subsystem."
  "There once were two watchmakers, named Hora and Tempus, who manufactured very fine watches. Both of them were highly regarded, and the phones in their workshops rang frequently -new customers were constantly calling them. However, Hora prospered, while Tempus became poorer and poorer and finally lost his shop. what was the reason?"
  "The watches the men made consisted of about 1,000 parts each. Tempus had so constructed his that if he had one partly assembled and had to put it down-to answer the phone say-it immediately fell to pieces and had to be reassembled from the elements. The better the customers liked his watches, the more they phoned him, the more difficult it became for him to find enough uninterrupted time to finish a watch."
  "The watches that Hora made were no less complex than those of Tempus. But he had designed them so that he could put together subassemblies of about ten elements each. Ten of these subassemblies, again, could be put together into a larger subassembly; and a system of ten of the latter subassemblies constituted the whole watch. Hence, when Hora had to put down a partly assembled watch in order to answer the phone, he lost only a small part of his work, and he assembled his watches in only a fraction of the man-hours it took Tempus."


• Mark Edwards, A Brief History of Holons.


• Timothy F. Allen, A SUMMARY OF THE PRINCIPLES OF HIERARCHY THEORY


• Yiyu Yao, Integrative levels of granularity, in: A. Bargiela and W. Pedrycz (Eds.), Human-Centric Information Processing Through Granular Modelling, Springer-Verlag, Berlin, pp. 31-47, 2009.



Philosophical foundations

• Wikipedia, Discourse on the Method.
• Rene Descarte, Discourse on the Method of Rightly Conducting one's Reason and Seeking Truth in the Sciences.
  "Now, just as a state is much better governed when it has only a few laws that are strictly obeyed than when it has a great many laws that can provide an excuse for vices, so I thought that in place of the large number of rules that make up logic I would find the following four to be sufficient, provided that I made and kept to a strong resolution always to obey them.
  (1) The first was never to accept anything as true if I didn't have evident knowledge of its truth: that is, carefully to avoid jumping to conclusions and preserving old opinions, and to include in my judgments only what presented itself to my mind so vividly and so clearly that I had no basis for calling it in question.
  (2) The second was to divide each of the difficulties I examined into as many parts as possible and as might be required in order to resolve them better.
  (3) The third was to direct my thoughts in an orderly manner, by starting with the simplest and most easily known objects in order to move up gradually to knowledge of the most complex, and by stipulating some order even among objects that have no natural order of precedence. (4) And the last ·was· to make all my enumerations so complete, and my reviews so comprehensive, that I could be sure that I hadn't overlooked anything."


• Wikipedia, Systems thinking.



Methodological foundations

• Wikipedia, How to Solve It. (A summary of the book by George Polya.)
• George Polya, How to solve it.
  "Decomposing and recombining are important operations of the mind."
  "If you go into detail you may lose yourself in details. Too many or too minute particulars are a burden of the mind. The may prevent you from giving sufficient attention to the main point, or even from seeing the main point at all. Think of the man who cannot see the forest for the trees."
  "Therefore, let us, first of all, understand the problem as a whole. Having understood the problem, we shall be in a better position to judge which particular points may be the most essential. Having examined one or two essential points we shall be in a better to position to judge which further details might deserve closer examination. Let us go into detail and decompose the problem gradually, but not further than we need to."


• Wikipedia, Structured programming.


• Niklaus Wirth, Program Development by Stepwise Refinement, Communications of the ACM, Vol. 14, No. 4, April 1971, pp. 221-227.
  "Program construction consists of a sequence of refinement steps. In each step a given task is broken up into a number of subtasks. Each refinement in the description of a task may be accompanied by a refinement of the description of the data which constitute the means of communication between the subtasks. Refinement of the description of program and data structures should proceed in parallel."
  "During the process of stepwise refinement, a notation which is natural to the problem in hand should be used as long as possible. "


• Wikipedia, Top-down and bottom-up design.


• Donald E. Knuth, Literate Programming, The Computer Journal (British Computer Society) 27 (2): 97-111.
  "When I first began to work with the ideas that eventually became the WEB system, I thought that I would be designing a language for "top-down" programming, where a top-level description is given first and successively refined. On the other hand I knew that I often created major parts of programs in a "bottom-up" fashion, starting with the definitions of basic procedures and data structures and gradually building more and more powerful subroutines. I had the feeling that top-down and bottom-up were opposing methodologies: one more suitable for program exposition and the other more suitable for program creation."
  "Top-down programming gives you a strong idea of where you are going, but it forces you to keep a lot of plans in your head; suspense builds up because nothing is really nailed down until the end. Bottom-up programming has the advantage that you continually wield a more and more powerful pencil, as more and more subroutines have been constructed; but it forces you to postpone the overall program organization until the last minute, so you might flounder aimlessly."


• E.W.Dijkstra
  • On Mathematical Methodology
    • Honors course "Mathematical Methodology", Spring 1996 (EWD1220)
      "This course is not about mathematical results but about doing mathematics, about wasting neither your own time, nor the time of your readers. Mathematics will be treated as the art and science of effective reasoning, the latter encompassing the design, recording and explanation of arguments."
    • Essays on the nature and role of mathematical elegance (1) (EWD619).
      "The teaching of effective thinking is certainly the teaching of an ability, of a methodology, and the first effect of teaching a methodology --rather than disseminating knowledge-- is that of enhancing the capacities of the already capable, thus magnifying the difference in intelligence."
  • On Using Structures
    • On useful structuring (EWD245).
      "I tend to think of the program consisting of a set of hierarchical layers, performing in steps the transition from what we have got into what we should like to have. The right of existence of these separate layers is that in each layer an independent abstraction is implemented: an identified choice is condensed in its coding."
    • On a methodology of design.
      "As soon as programming emerges as a battle against unmastered complexity, it is quite natural that one turns to that mental discipline whose main purpose has been since centuries to apply effective structuring to otherwise unmastered complexity. That mental discipline is more or less familiar to all of us, it is called Mathematics. "
  • On Conscious Efforts
    • On useful structuring (EWD245).
      "The purpose of this minor contribution is to stress the urgency to make a conscious effort at exploiting "structure" as a useful thinking aid. Furthermore it gives some of the conclusions such an effort has led me to so far. "
    • Some beautiful arguments using mathematical induction (EWD 697).
      "Finally I hope to demonstrate how a conscious effort at applying mathematical induction can be of great heuristic value."
    The structure of the 'THE'-multiprogramming system (EWD196).
    "Be aware of the fact that experience does by no means automatically lead to wisdom and understanding, in other words: make a conscious effort to learn as much as possible from your precious experiences."
    • Visuals for BP's Venture Research Conference (EWD963).
      "We have learnt that programming methodology and mathematical methodology in general are not so far apart at all. (For instance, a conscious separation of concerns is equally valuable for both.)"
    • A preliminary investigation into Computer Assisted Programming.
      "... my claim is that when one makes a conscious effort at it, it is more feasible than trying to get a program correct when the problem of correctness has been left to the debugging phase." my claim is that when one makes a conscious effort at it, it is more feasible than trying to get a program correct when the problem of correctness has been left to the debugging phase.
    • On the role of scientific thought (EWD447).
      "Scientific thought comprises "intelligent thinking" as described above. A scientific discipline emerges with - the usually rather slow! - discovery of which aspects can be meaningfully "studied in isolation for the sake of their own consistency", in other words: with the discovery of useful and helpful concepts. Scientific thought comprises in addition the conscious search for the useful and helpful concepts."



Computational foundations

• Bargiela, A., Pedrycz, W., Granular Computing: An Introduction, Kluwer Academic Publishers, Boston, 2002. Chapter 1.


• O.G. Selfridge, Pandemonium: A paradigm for learning. In D. V. Blake and A. M. Uttley, editors, Proceedings of the Symposium on Mechanisation of Thought Processes, pages 511-529, London, 1959.


• Yiyu Yao and Jigang Luo, Top-Down Progressive Computing.



Further Readings

• Yiyu Yao, Artificial Intelligence Perspectives on Granular Computing, Granular Computing and Intelligent Systems Intelligent Systems Reference Library, 2011, Volume 13, 17-34.


• J.M. Wing, Computational thinking.


• J. Kramer, Is abstraction the key to computing?


• Wikipedia, Unix philosophy.
• Eric Steven Raymond, The Art of Unix Programming

PROLOG Programming for AI

February 15, 17, 2012

• Textbook, Chapter 2 (review): Logic foundations for PROLOG.
• Textbook, Chapter 3 (review): Foundations of PROLOG Interpreter.
• Textbook, Section 14.3.
• For binary resolution, read Section 14.2.
• For the implemenation of PROLOG using binary resolution and serach,, read Section 14.3.

February 27, 29, March 2, 5, 7, 9, 2012

• PROLOG Programming
• State Space Search using PROLOG

Natural Language Processing

March 12, 14, 16, 19, 2012

• Chapter 15, Textbook


• Wikipedia, Zipf's law.
• H.P. Luhn, The Automatic Creation of Literature Abstracts
• C.J. van Rijsbergen, Information Retrieval, Chapter 2, AUTOMATIC TEXT ANALYSIS


• Wikipedia, Chomsky hierarchy

March 21, 23, 2012: Natural language processing using PROLOG

• David S. Warren, Grammars in Prolog


• Amzi! inc. Definite Clause Grammars (DCGs)


• Tim Smith, Artificial Intelligence Programming in Prolog (AIPP) Lecture 13. Sentence Manipulation.

Neural Networks

March 26, 30, 2012:

• Sections 11.1, 11.2
• Wikipedia, Perceptron
• Raúl Rojas Perceptron Learning

Expert Systems and Naive Bayesian Classification

March 30, April 2, 4, 9, 11, 2012:

• Sections 8.1 and 8.2
• Sections 5.1, 5.2, 5.3, 5.4
• Wikipedia, Naive Bayes classifier
• Andrew Moore, A Short Intro to Naive Bayesian Classifiers

Additional readings (for interest only)