INTELLIGENT SEARCH AGENTS

BY DARREN GREAVES

ACKNOWLEDGEMENTS
My Project Supervisor:
Mike Van Kleef

Ex-colleagues from my placement time at Braid Systems:
Bryan Childs
Sean Batten

Someone who pushed me to work hard all the time:
Christine


1. ABSTRACT
2. INTRODUCTION
2.1. OVERVIEW OF PROBLEM DOMAIN.
2.2.POSSIBLE SOLUTIONS.
2.2.1.Web Browsing
2.2.2. Search Engines
2.2.3. Intelligent Search Agents
2.2.4. Categories Of Agents
2.2.4.1. Mobile Agents
2.2.4.2. Non-Mobile Agents
2.3. MY SOLUTION TO THE PROBLEM
3. RESEARCH
3.1. INTRODUCTION
3.2. AGENT RELATED PROJECTS
3.2.1.Large Scale Projects
3.2.1.1. Harvest
3.2.1.2. Harvest Evaluation
3.2.1.3. ARPA Knowledge Sharing Effort[ARP WWW]
3.2.1.4. ARPA Evaluation
3.2.2. Agent Programming Languages & Paradigms
3.2.2.1. KQML Knowledge Query and Manipulation Language
3.2.2.2. KQML Evaluation
3.2.2.3. Agent-Oriented Programming Paradigm
3.2.2.4. AOPP Evaluation
3.2.2.5. LALO Langage D'agents Logiciel Objet
3.2.2.6. LALO Evaluation
3.2.3. Downloadable Agent Development Toolkits
3.2.3.1. Agent Building Environment (ABE)
3.2.3.2. ABE Evaluation
3.2.3.3. Aglets Self-contained Java Objects
3.2.3.4. Aglets Evaluation
3.3. CONCLUSIONS
4. IMPLEMENTATION OVERVIEW
4.1. INTRODUCTION
4.2. PLATFORM
4.3. CHOICE OF LANGUAGE
4.3.1. Languages Available
4.3.2. Criteria for Evaluation
4.3.3. Final Choice
4.4. CHOICE OF METHODOLOGY/DEVELOPMENT STRATEGY
4.4.1. Methodologies Available
4.4.2. Criteria for Evaluation
4.4.3. Final Choice
4.5. DESIGN
4.5.1.1. Introduction
4.5.1.2. Perform Searches On Internet/Intranet Content
4.5.1.3. Add New Filters To Perform Different Searches
4.5.2. Design One
4.5.3. Evaluation of Design One
4.5.4. Design Two
4.5.5. Evaluation of Design Two
5. IMPLEMENTATION DETAILS
5.1. INTRODUCTION
5.2. PROTOTYPE ONE
5.2.1. Introduction
5.2.2. Program Creation Using MSVC
5.2.3. Creation of User Interface
5.2.4. Creation of Member Functions
5.2.5. Build a Class Structure
5.2.6. Development of Search Engine
5.2.6.1. Read pages off Web Server
5.2.6.2. MFC Internet Classes
5.2.6.3. Good Manners
5.2.6.4. Reading the File
5.2.7. Multithreaded code
5.2.7.1. Threads in MFC
5.2.7.2. Choice of Implementation
5.2.8. Parsing HTML
5.2.9. Evaluation of Prototype One
5.3. PROTOTYPE TWO
5.3.1. Introduction
5.3.2. Problems and Solutions
5.3.2.1. Program hung when searching specific sites.
5.3.2.2. User interface was not updated while search took place.
5.3.2.3. Search results needed to be posted while the search was ongoing.
5.3.2.4. A sample filter needed to be implemented.
5.3.2.4.1. ActiveX Filters
5.3.2.4.2. Implementing the Filter
5.3.2.5. Re-design of front-end so that filters were chosen from a drop-down list.
5.3.3. Evaluation of Prototype Two
5.4. PROTOTYPE THREE
5.4.1. Introduction
5.4.2. Problems and Solutions
5.4.2.1. A Major Rewrite Required
5.4.2.2. Multithreaded Implementation Problems
5.4.2.3. Program still hanging on certain sites
5.4.2.3.1. Searching the Internet
5.4.2.3.2. Tracing Code
5.4.3. Completion of Implementation
6. FUTURE WORK
6.1. INTRODUCTION
6.2. ADDITIONAL FEATURES
6.2.1. Off-line database
6.2.2. Web Interface To Program
6.3. FEATURES THAT SHOULD HAVE BEEN IMPLEMENTED
6.3.1. Search Multiple Sites
6.3.2. Improved Separation of Different Parts of Program
6.3.3. ActiveX Filters
6.3.4. Known Bugs
6.3.4.1. Release Build Crashing
6.4. IDEAS FOR SEARCH FILTERS
6.4.1. Introduction
6.4.2. Synonym Filter
6.4.3. Graphic Filter
7. EVALUATION
7.1. ACHIEVEMENTS
7.1.1. A working program
7.1.2. To learn about the Intelligent Agent Paradigm
7.1.3. To learn more about ActiveX
7.1.4. To practice and improve my programming skills
7.2. MY THOUGHTS ON THE PROJECT
7.2.1. how to manage my time better
7.2.2. undertaking effective research and drawing some meaningful conclusions
7.2.3. how to be critical in my writing
7.2.4. remaining focused on the project
7.3. METHODOLOGIES
8. CONCLUSION
9. APPENDICES
9.1. APPENDIX A - TECHNICAL BACKGROUNDERS
9.1.1. ActiveX and other Component Technologies
9.1.1.1. Component Technology.
9.1.1.2. OLE
9.1.1.3. COM
9.1.1.4. DCOM
9.1.1.5. ActiveX
9.1.1.5.1. ActiveX Controls.
9.1.1.5.2. ActiveX Containers.
9.1.1.5.3. Usage
9.1.2. Multithreaded Processing.
9.1.3. Apache Web Server
9.2. APPENDIX B - REFERENCES & BIBLIOGRAPHY
9.2.1. World Wide Web references
9.2.2. Journal References
9.2.3. Bibliography
9.3. APPENDIX C - INDICES
9.3.1. Index of Images
9.3.2. Index of Tables
9.4. APPENDIX D - GLOSSARY