CAP 5636 - Advanced Artificial Intelligence

Fall 2024

Course description:	Principles of artificial intelligence. Uninformed and informed search. Constraint satisfaction. AI for game playing. Probabilistic reasoning, Markov decision processes, hidden Markov models, Bayes nets. Neural networks and deep learning.
Student learning outcomes:	By the end of the semester the students will be able to: understand the search and decision making techniques used in modern artificial intelligence apply artificial intelligence techniques in their own code understand the societal and ethical implications of artificial intelligence
Instructor:	Dr. Lotzi Bölöni
Office Location:	HEC - 319
E-mail:	Ladislau.Boloni@ucf.edu (preferred means of communication)
Team:	TA: Kunyang Li kunyang.li@ucf.edu Grader: Deepak Kumar Kunda deepakkumar.kunda@ucf.edu
Web Site:	http://www.cs.ucf.edu/~lboloni/Teaching/CAP5636_Fall2024/index.html The assignments and the other announcements will be posted on the course web site
Classroom:	HEC-118
Class hours:	Tue, Th 12:00pm - 1:15pm
Office hours:	Tue, Th 1:30pm - 3:00pm (in HEC 319)
Enrollment requirements:	CAP 4630, or consent of instructor.
Required texts:	There is no required textbook.
Recommended readings:	Stuart Russel and Peter Norvig, Artificial Intelligence - A Modern Approach, 4th edition

Syllabus

Date	Topic	Lecture Notes, Readings, Homeworks
Tue, Aug. 20	Introduction and content of the class The agent view of AI. Topics covered by the AI and ML classes. History and positioning of AI Motivating AI. Dangers of AI and AGI. Early history Expert systems	[slides] Content of the class [slides] History and positioning of AI [homework] HW1: AI History and Future - Due Aug. 28, 2024 [reading] Perceptrons (New York Times, July 13, 1958) [reading] Nick Bostrom - How long before superintelligence? (1998)
Thu, Aug. 22	History and positioning of AI Neural networks The two intellectual traditions: logic vs neural networks A melting pot of other ideas The agent view of AI
Tue, Aug. 27	Planning with uninformed search Reflex agents Search problems	[slides] Uninformed search
Thu, Aug. 29	Planning with uninformed search (cont'd) Depth first and breadth first search Uniform cost search	[homework] HW2: Intro to deep learning - Due Sept. 18, 2024
Tue, Sept. 3	*Planning with informed search: A search and heuristics** Informed search methods Heuristics Greedy search A* search Graph search	[slides] Informed search
Thu, Sep. 5	Game playing and adversarial search Adversarial search, minimax Alpha Beta pruning Types of games	[slides] Adversarial search
Tue, Sep. 10	Class cancelled due to fire alarm.
Thu, Sep. 12	Game playing and adversarial search (cont'd) The problem of depth Evaluation functions Monte Carlo Tree Search
Tue, Sep. 17
Thu, Sept. 19	Expectimax search and utilities Expectimax search Refresher about probabilities Utilities and rationality	[slides] Expectimax search and utilities
Tue, Sep. 24	Expectimax search and utilities (cont'd) Utilities Axioms of rationality	[homework] HW3: Searching for plans - Due October 15
Thu, Sept. 26	Class cancelled due to Hurricane Helene.
Tue, Oct. 1	Midterm 1 - Introduction to Expectimax (not including utilities)
Thu, Oct. 3	Markov decision processes Defining MDPs: policies and utilities Optimal policy, value of state, value of Q-state	[slides] Markov Decision Processes
Tue, Oct. 8	Class cancelled due to Hurricane Milton.
Thu, Oct. 10	Class cancelled due to Hurricane Milton.
Tue, Oct. 15	Probability Random variables Joint and marginal distributions, conditional distribution class taught by TA Kunyang Li	[slides] Probabilities - Introduction
Thu, Oct. 17	Probability (cont'd) Product rule, chain rule, Bayes' rule Inference Independence class taught by TA Kunyang Li
Tue, Oct. 22	Markov decision processes 2 Value iteration
Thu, Oct. 24	Policy extraction Policy iteration
Tue, Oct. 29	Reinforcement learning Reinforcement learning as a twist on MDPs Model-based and model-free learning Temporal difference learning	[slides] Reinforcement learning
Thu, Oct. 31	Reinforcement learning (cont'd) Exploration vs. exploitation, regret Generalization across states
Tue, Nov. 5	Deep reinforcement learning Challenges with Q estimation The "deadly triad" DQN Double Q-learning	[slides] Deep reinforcement learning [homework] HW4 - MDP and Q-Learning - Due November 21 [homework] HW5 - DQN - Due November 28 [homework] HW6 - Societal implications - Due November 28
Thu, Nov. 7	Midterm 2: from Expectimax to Reinforcement Learning (inclusive)
Tue, Nov. 12	Imitation learning	[slides] Imitation learning
Thu, Nov. 14	Independent variables and Bayes' nets Independent random variables Conditional independence Bayes' nets	[slides] Independence
Tue, Nov. 19	Hidden Markov models Hidden Markov models Example: robot localization	[slides] Hidden Markov Models
Thu, Nov 21	Particle filters and applications of HMMs Particle filters Robot localization with particle filters	[slides] Particle filters and Applications of HMMs
Tue, Nov. 26	Artificial General Intelligence Intelligence tests (Turing, Loebner prize, Winograd schema) Artificial Superintelligence Alignment problems LLMs Ethical and societal implications of AI Ethical issues Economic impact Existential threat
Thu, Dec. 5	Final exam: Thursday December 5, 2024, 10:00am - 12:50pm