Research Bot Information

About the Bot

The MichaelDistelResearchBot (version 1.0) is a web crawler that helps gather information for research purposes. This bot operates under the following user agent string:

Allow with Restrictions

MichaelDistelResearchBot/1.0 (+https://michaeldistel.com/bot-info)

Purpose

MichaelDistelResearchBot/1.0 is a small-scale, academic research crawler operated by Michael Distel. The bot's mission is to gather publicly available, non-personal data from the web to support a research project on economics within the venture capital (VC) ecosystem.

Research Focus Areas

• Identifying who participates in the VC space
• Understanding the primary areas of focus across different VC firms
• Estimating the number of active VC firms globally
• Tracking general metrics such as investment trends, sector focus, and fund sizes

Ethical Framework

The bot operates within a strictly ethical framework that includes:

• Respecting robots.txt directives
• Avoiding login-restricted or paywalled content
• Ensuring low impact on website performance through rate limiting
• Collecting only publicly available, non-personal data

Robots.txt Configuration

Website owners can control this bot's access using their robots.txt file. Here are example configurations:

Allow with Restrictions

User-agent: MichaelDistelResearchBot
Disallow: /private-data/
Allow: /

Block Completely

User-agent: MichaelDistelResearchBot
Disallow: /

Privacy & Opt-Out

MichaelDistelResearchBot only collects publicly available data related to the venture capital industry.

If you would like to request exclusion of your website from this research, you can:

• Add a Disallow rule for MichaelDistelResearchBot in your robots.txt (see configuration above), or
• Contact Michael Distel directly at [email protected] with your request.

All opt-out requests will be respected and promptly acted upon.

Contact Information

If you have any questions or concerns about this bot's behavior, please contact me through: [email protected]