{"repo":"oxylabs/oxylabs-ai-studio-py","free":true,"listed":false,"github":"https://github.com/oxylabs/oxylabs-ai-studio-py","clone":"git clone https://github.com/oxylabs/oxylabs-ai-studio-py.git","description":"Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering. ","language":"Python","stars":3295,"topics":["ai-crawler","ai-grounding","ai-scraper","ai-scraping","ai-search","ai-tools","ai-training","ai-web-scraper","firecrawl-alternative","proxy-scraper","python-ai","search-api","tavily-alternative","web-scraping","web-scraping-ai","web-scraping-api","web-scraping-python","web-search-api"],"license":"MIT","category":"scraper","readme_excerpt":"# OxyLabs AI Studio Python SDK\n\n[![AI-Studio Python (1)](https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/Ai-Studio2.png)](https://aistudio.oxylabs.io/?utm_source=877&utm_medium=affiliate&utm_campaign=ai_studio&groupid=877&utm_content=ai-studio-js-github&transaction_id=102f49063ab94276ae8f116d224b67) \n\n[![](https://dcbadge.limes.pink/api/server/Pds3gBmKMH?style=for-the-badge&theme=discord)](https://discord.gg/Pds3gBmKMH) [![YouTube](https://img.shields.io/badge/YouTube-Oxylabs-red?style=for-the-badge&logo=youtube&logoColor=white)](https://www.youtube.com/@oxylabs)\n\nA simple Python SDK for seamlessly interacting with [Oxylabs AI Studio API](https://aistudio.oxylabs.io/) services, including AI-Scraper, AI-Crawler, AI-Browser-Agent and other data extraction tools.\n\n## Requirements\n- python 3.10 and above\n- API KEY\n\n## Installation\n\n```bash\npip install oxylabs-ai-studio\n```\n\n## Usage\n\n### Crawl (`AiCrawler.crawl`)\n\n```python\n\nfrom oxylabs_ai_studio.apps.ai_crawler import AiCrawler\n\ncrawler = AiCrawler(api_key=\"<API_KEY>\")\n\nurl = \"https://oxylabs.io\"\nresult = crawler.crawl(\n    url=url,\n    user_prompt=\"Find all pages with proxy products pricing\",\n    output_format=\"markdown\",\n    render_javascript=False,\n    return_sources_limit=3,\n    geo_location=\"United States\",\n)\nprint(\"Results:\")\nfor item in result.data:\n    print(item, \"\\n\")\n\n```\n\n**Parameters:**\n- `url` (str): Starting URL to crawl (**required**)\n- `user_prompt` (str): Natural language prompt to guide extraction (**required**)\n- `output_format` (Literal[\"json\", \"markdown\", \"csv\", \"toon\"]): Output format (default: \"markdown\")\n- `schema` (dict | None): Json schema for structured extraction (required if output_format is \"json\", \"csv\" or \"toon\")\n- `render_javascript` (bool): Render JavaScript (default: False)\n- `return_sources_limit` (int): Max number of sources to return (default: 25)\n- `geo_location` (str): Proxy location in ISO2 format or country canonical name. See [docs](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/features/localization/proxy-location#list-of-supported-geo_location-values)\n- `max_credits` (int | None): Maximum of credits to use (optional)\n\n### Scrape (`AiScraper.scrape`)\n\n```python\nfrom oxylabs_ai_studio.apps.ai_scraper import AiScraper\n\nscraper = AiScraper(api_key=\"<API_KEY>\")\n\nschema = scraper.generate_schema(prompt=\"want to parse developer, platform, type, price game title, genre (array) and description\")\nprint(f\"Generated schema: {schema}\")\n\nurl = \"https://sandbox.oxylabs.io/products/3\"\nresult = scraper.scrape(\n    url=url,\n    output_format=\"json\",\n    schema=schema,\n    render_javascript=False,\n    optimize_content=True,\n)\nprint(result)\n\n```\n**Parameters:**\n- `url` (str): Target URL to scrape (**required**)\n- `output_format` (Literal[\"json\", \"markdown\", \"csv\", \"screenshot\", \"toon\"]): Output format (default: \"markdown\")\n- `schema` (dict | None): JSON schema for structured extraction (required if output_format is \"json\", \"csv\" or \"toon\")\n- `render_javascript` (bool | string): Render JavaScript. Can be set to \"auto\", meaning the service will detect if rendering is needed (default: False)\n- `geo_location` (str): Proxy location in ISO2 format or country canonical name. See [docs](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/features/localization/proxy-location#list-of-supported-geo_location-values)\n- `user_agent` (str): User-Agent request header. See more at https://developers.oxylabs.io/scraping-solutions/web-scraper-api/features/http-context-and-job-management/user-agent-type.\n- `optimize_content` (bool): Return cleaner markdown by focusing on the main page content. Output will be smaller in size when set to True (default: True)\n- `browser_instructions` (list[BrowserInstruction] | None): Browser actions to run before capture (click, input, wait, etc.). Requires `render_javascript=True`. Format follows [Web Scraper API browser instructions](https://developers.oxylabs.io/products/web-scraper-api/features/js-rendering-and-browser-control#browser-instructions).\n\n### Browser Agent (`BrowserAgent.run`)\n\n```python\nfrom oxylabs_ai_studio.apps.browser_agent import BrowserAgent\n\nbrowser_agent = BrowserAgent(api_key=\"<API_KEY>\")\n\nschema = browser_agent.generate_schema(\n    prompt=\"game name, platform, review stars and price\"\n)\nprint(\"schema: \", schema)\n\nprompt = \"Find if there is game 'super mario odyssey' in the store. If there is, find the price. Use search bar to find the game.\"\nurl = \"https://sandbox.oxylabs.io/\"\nresult = browser_agent.run(\n    url=url,\n    user_prompt=prompt,\n    output_format=\"json\",\n    schema=schema,\n)\nprint(result.data)\n```\n\n**Parameters:**\n- `url` (str): Starting URL to browse (**required**)\n- `user_prompt` (str): Natural language prompt for extraction (**required**)\n- `output_format` (Literal[\"json\", \"markdown\", \"html\", \"screenshot\", \"csv\", \"toon\"]): Output format (default: \"markdown\")\n- `schema` (dict | None): Json schema for structured extraction (required if output_format is \"json\", \"csv\" or \"toon\")\n- `geo_location` (str): Proxy location in ISO2 format or country canonical name. For example 'Germany' (capitalized).\n\n### Search (`AiSearch.search`)\n\n```python\nfrom oxylabs_ai_studio.apps.ai_search import AiSearch\n\n\nsearch = AiSearch(api_key=\"<API_KEY>\")\n\nquery = \"lasagna recipe\"\nresult = search.search(\n    query=query,\n    limit=5,\n    render_javascript=False,\n    return_content=True,\n)\nprint(result.data)\n\n# Or for fast search\nresult = search.instant_search(\n    query=query,\n    limit=10,\n)\nprint(result.data)\n```\n\n**Parameters:**\n- `query` (str): What to search for (**required**)\n- `limit` (int): Maximum number of results to return (default: 10, maximum: 50)\n- `render_javascript` (bool): Render JavaScript (default: False)\n- `return_content` (bool): Whether to return markdown contents in results (default: True)\n- `geo_location` (*string*): ISO 2-letter format, country name, coordinate formats are supported. See more at [SERP Localization](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/features/localization/serp-localization).\n\n> **Note:** When `limit <= 10` and `return_content=False`, the search automatically uses the instant endpoint (`/search/instant`) which returns results immediately without polling, providing faster response times.\n\nInstant search supported parameters:\n- `query` (*string*): The search query.\n- `limit` (*integer*): The maximum number of search results to return. Maximum: 10.\n- `geo_location` (*string*): Google's canonical name of the location. See more at [Google Ads GeoTargets](https://developers.google.com/google-ads/api/data/geotargets).\n\n\n### Map (`AiMap.map`)\n```python\nfrom oxylabs_ai_studio.apps.ai_map import AiMap\n\n\nai_map = AiMap(api_key=\"<API_KEY>\")\npayload = {\n    \"url\": \"https://career.oxylabs.io\",\n    \"search_keywords\": [\"career\", \"jobs\", \"vacancy\"],\n    \"user_prompt\": \"job ad pages\",\n    \"max_crawl_depth\": 2,\n    \"limit\": 10,\n    \"geo_location\": \"Germany\",\n    \"render_javascript\": False,\n    \"include_sitemap\": True,\n    \"max_credits\": None,\n    \"allow_subdomains\": False,\n    \"allow_external_domains\": False,\n}\nresult = ai_map.map(**payload)\nprint(result.data)\n```\n**Parameters:**\n- `url` (str): Starting URL or domain to map (**required**)\n- `search_keywords` (list[str]): Keywords for URLs paths filtering (default: None)\n- `user_prompt` (str | None): Natural language prompt for keyword search. Can be used together with 'search_keywords' or standalone (optional)\n- `max_crawl_depth` (int): Max crawl depth (1..5, default: 1)\n- `limit` (int): Max number of URLs to return (default: 25)\n- `geo_location` (str): Proxy location in ISO2 format or country canonical name. See [docs](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/features/localization/proxy-location#list-of-supported-geo_location-values)\n- `render_javascript` (bool): JavaScript rendering (default: False)\n- `include_sitemap` (bool): Whether to include sitemap as seed (default: True)\n- `max_credits` (int | None): Maximum of credits to use (optional)\n- `allow_subdomains` (bool): Include subdomains (default: False)\n- `allow_external_domains` (bool): Include external domains (default: False)\n\n---\nSee the [examples](https://github.com/oxylabs/oxylabs-ai-studio-py/tree/main/examples) folder for usage examples of each method. Each method has corresponding async version.\n","default_branch":"main","files":34,"tree":[".github/workflows/publish.yml",".gitignore","Ai-Studio2.png","LICENSE","agentic_code_guide.md","examples/ai_map.py","examples/browser_agent.py","examples/crawl_generated_schema.py","examples/crawl_markdown.py","examples/crawl_pydantic_schema.py","examples/scrape_generated_schema.py","examples/scrape_markdown.py","examples/scrape_pydantic_schema.py","examples/search_instant.py","examples/search_no_content.py","examples/search_with_content.py","images/Github-AI-Studio-1262x525px new.png","images/Github-AI-Studio-1262x525px.png","images/banner1.png","makefile","pyproject.toml","readme.md","src/oxylabs_ai_studio/__init__.py","src/oxylabs_ai_studio/apps/ai_crawler.py","src/oxylabs_ai_studio/apps/ai_map.py","src/oxylabs_ai_studio/apps/ai_scraper.py","src/oxylabs_ai_studio/apps/ai_search.py","src/oxylabs_ai_studio/apps/browser_agent.py","src/oxylabs_ai_studio/client.py","src/oxylabs_ai_studio/logger.py","src/oxylabs_ai_studio/models.py","src/oxylabs_ai_studio/settings.py","src/oxylabs_ai_studio/utils.py","uv.lock"],"storefront":"/r/oxylabs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/oxylabs/oxylabs-ai-studio-py/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}