Taobao — Keyword Search

keyword + optional filters → paginated product listing (itemId, title, price, shop, image, sales)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Search Taobao/Tmall for products by keyword and extract product cards from search results pages.

Prerequisites

• Target page is already open in the browser: https://s.taobao.com/search
• User is logged in to Taobao (user avatar or nickname visible in the page header)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Taobao has been confirmed in the current session → skip this step.

Otherwise: open https://www.taobao.com and observe the page header:

• User nickname visible (e.g., "心林vs妞妞") → logged in, continue execution
• "亲，请登录" or login button visible → not logged in, inform the user that Taobao login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

DOM: product search results (data extraction)

Navigate to search results URL, then extract:

1. navigate "https://s.taobao.com/search?q={keyword}&page={page}&ie=utf8"
2. wait stable
3. eval "$(python scripts/search-products.py '{keyword}' --page {page} --sort '{sort}' --tab '{tab}' --start-price {startPrice} --end-price {endPrice})"

URL Parameters:

• q: URL-encoded keyword
• page: page number, 1-based, default 1
• sort: sort order — empty string (default/recommended), sale-desc (by sales), price-asc (price low→high), price-desc (price high→low)
• tab: mall for Tmall-only results; omit for all results
• startPrice / endPrice: price range filter in yuan (e.g., startPrice=100&endPrice=500)

Output example:

[
  {
    "itemId": "694593508978",
    "itemUrl": "https://item.taobao.com/item.htm?id=694593508978",
    "title": "蓝牙耳机2025新款官方",
    "subTitle": "AI耳机热卖榜第1名",
    "priceYuan": 79.9,
    "priceDesc": "券后价",
    "imageUrl": "https://img.alicdn.com/imgextra/...",
    "salesCount": "40万+人付款",
    "shopName": "金运旗舰店",
    "location": "广东",
    "rating": null,
    "tags": ["政府补贴15%", "官方立减26元"]
  }
]

Notes:

• priceYuan is the displayed price (may be post-coupon price, not pre-coupon)
• priceDesc indicates price type: 券后价 (after coupon), 补贴价 (subsidized price), etc.
• rating is rarely shown on search cards; null is expected
• Sponsored/ad items have itemUrl pointing to click.simba.taobao.com — they will have itemId extracted from query params

Enum Parameters

[collection failed] sort values: confirmed values are empty string (default), sale-desc, price-asc, price-desc; no API for enumeration, values are hardcoded constants.

Pagination

URL Pagination: URL pattern https://s.taobao.com/search?q={keyword}&page={N}&ie=utf8, increment page from 1. Each page returns ~47 items. Termination: when result count < 10 or returned items duplicate previous page.

Success Criteria

result count >= 1 and itemId non-null rate = 100%

Known Limitations

• Requires Taobao login; unauthenticated sessions redirect to login page
• Prices shown are displayed prices (may be post-coupon), not pre-discount prices
• Sponsored/ad items appear at unpredictable positions in results
• Price filter (startPrice/endPrice) filters on pre-coupon prices, so post-coupon prices displayed may fall outside the requested range near boundaries
• Taobao may return different result counts across pages; page count is approximate

Execution Efficiency

• Batch orchestration: Write a bash script to loop through keywords serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Add 2–3 second intervals between page navigations. To increase throughput, open multiple stealth browser sessions and distribute keywords across them.
• Test before batch execution: After writing a batch script, you must first test with 1–2 keywords to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly.
• Reduce redundant pre-operations: When scraping multiple pages for one keyword, navigate page 2, 3 etc. within the same session without re-login checks.
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/taobao-keyword-search.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Taobao — Product Detail

itemId → full product detail (title, price, shop, images, SKU variants, attributes)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Navigate to a Taobao or Tmall product page and extract full product information from the DOM.

Prerequisites

• Target page is already open in the browser: https://item.taobao.com/item.htm?id={itemId}
• User is logged in to Taobao (user avatar or nickname visible in the page header)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Taobao has been confirmed in the current session → skip this step.

Otherwise: open https://www.taobao.com and observe the page header:

• User nickname visible → logged in, continue execution
• Login button visible → not logged in, inform the user that Taobao login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

DOM: product detail page (data extraction)

Navigate to the product page and extract all fields:

1. navigate "https://item.taobao.com/item.htm?id={itemId}"
2. wait stable
3. Close any popup if present: look for buttons with text "开心收下", "不了", "关闭" and click to dismiss
4. eval "$(python scripts/extract-product.py '{itemId}')"

Output example:

{
  "itemId": "744983869996",
  "itemUrl": "https://detail.tmall.com/item.htm?id=744983869996",
  "isTmall": true,
  "title": "绿联转换插头英标马来西亚新加坡澳洲韩国新西兰Switch插头转换器",
  "price": 17.9,
  "priceFormatted": "￥17.9",
  "originalPrice": null,
  "shopName": "绿联数码旗舰店",
  "shopUrl": "https://shop67095450.taobao.com/category.htm",
  "shopId": "67095450",
  "images": [
    "https://img.alicdn.com/imgextra/i3/713464357/O1CN01fQN7GG1i3YdCfBjzF_!!0-item_pic.jpg"
  ],
  "skuVariants": [
    "磨砂黑|英转中转换器【适用国内电器】适用马来西亚/新加坡等国家",
    "轻巧白|英转中转换器【适用国内电器】适用马来西亚/新加坡等国家"
  ],
  "attributes": {
    "产地": "中国大陆",
    "品牌": "绿联",
    "转换器类型": "英标",
    "型号": "S510"
  },
  "reviewCount": "7000+"
}

Notes:

• isTmall: true when product is on Tmall (URL contains tmall.com)
• price: the currently displayed price (may be flash sale, subsidized, or post-coupon price); multiply SKU variants affect the displayed price
• originalPrice: the crossed-out original price when a sale is active; null when no sale
• shopName: cleaned shop name (shop header link text)
• images: deduplicated, .webp suffix removed for cleaner URLs; first image is the main listing image
• skuVariants: all visible SKU option labels (color, size, etc.)
• attributes: key-value pairs from the product specifications section; may include 颜色分类 with all variant names as a combined string
• reviewCount: approximate text from page (e.g., "7000+"), not a precise integer

Error handling: if title is null, the product page may not have loaded correctly — check if still on the product page (state to inspect URL) and retry navigation.

Pagination

N/A — single product page, no pagination.

Success Criteria

title non-null AND itemId matches input

Known Limitations

• price is the currently displayed price for the default/first SKU; to get prices for other SKU variants, click each skuVariants option and re-run the price extraction
• Shop name in raw DOM concatenates rating text; the script extracts only the link text but may still include ratings in some layouts
• Images include both product listing images and some thumbnail duplicates; the script deduplicates by URL
• originalPrice extraction depends on the subPrice-- class structure which varies by product type (flash sale vs regular discount)

Execution Efficiency

• Batch orchestration: Write a bash script to loop through itemIds serially within a single session; add 2–3 second intervals between navigations.
• Test before batch execution: After writing a batch script, you must first test with 1–2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly.
• Reduce redundant pre-operations: When scraping multiple products, stay in the same session without re-login checks between items.
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/taobao-product-detail.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Taobao — Product Reviews

itemId → paginated customer reviews (reviewer, date, purchased SKU, text, photos)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Navigate to a Taobao/Tmall product page, load the reviews section, and extract customer review content.

Prerequisites

• Target page is already open in the browser: https://item.taobao.com/item.htm?id={itemId}
• User is logged in to Taobao (user avatar or nickname visible in the page header)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Taobao has been confirmed in the current session → skip this step.

Otherwise: open https://www.taobao.com and observe the page header:

• User nickname visible → logged in, continue execution
• Login button visible → not logged in, inform the user that Taobao login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

DOM: product reviews (data extraction)

The reviews section is lazy-loaded below the main product area. Follow these steps to load and extract reviews:

1. navigate "https://item.taobao.com/item.htm?id={itemId}"
2. wait stable
3. Close any popup: look for buttons with text "开心收下", "不了", "关闭" and click to dismiss
4. Scroll to trigger lazy loading of the tabs/reviews section: scroll down --amount 8000
5. wait --selector "[class*='tabTitleItem--']" --state attached --timeout 10000
   • If timeout: scroll down --amount 8000 again and retry wait once more
   • If still no tabs after 2 attempts: take screenshot to confirm page state; the product page may be rendering in a condensed mode — check Known Limitations below
6. eval "$(python scripts/extract-reviews.py '{itemId}')"

Output example:

[
  {
    "username": "一笑奈何",
    "date": "2026-06-03",
    "purchasedSku": "轻巧白|英转中转换器【适用国内电器】适用马来西亚/新加坡等国家",
    "content": "商品非常好，造工很用心！，还会再回购！",
    "photos": [
      "https://gw.alicdn.com/bao/uploaded/i1/O1CN015Cyg4b2FPR2YNq3PD_!!4611686018427383816-0-rate.jpg"
    ],
    "rating": null
  }
]

Notes:

• purchasedSku: the specific variant the reviewer purchased (extracted from "已购：{sku}" prefix in review header)
• content: review text body; may be empty if reviewer submitted only photos
• photos: review photo URLs; empty array if no photos
• rating: star rating; not always visible in current page layout (null is common)
• Reviews shown are the default sort (most recent or most helpful as determined by Taobao)

Error handling: if result count = 0 after scroll attempts, the reviews section may not have loaded in the current browser rendering environment. Try navigating to the product page fresh (navigate again) and repeating the scroll sequence. If still failing, this is a known rendering limitation — see Known Limitations below.

DOM: paginate to next review page

After extracting current page reviews:

1. eval "$(python scripts/next-review-page.py)"
   • Returns {"hasNext": true, "buttonText": "下一页"} if next page exists, or {"hasNext": false} if on last page
2. If hasNext is true: state to find the "下一页" button index → click <index>
3. wait stable
4. Re-run eval "$(python scripts/extract-reviews.py '{itemId}')"

Enum Parameters

[collection failed] Sort/filter options for reviews (e.g., newest, most helpful): these controls exist in the reviews section UI but require the tabs section to be loaded; their URL parameters are not exposed and must be set via UI clicks on the sort tabs within the reviews section.

Pagination

DOM Pagination: Click the "下一页" button in the reviews section footer. Each page shows ~10 reviews. Termination: "下一页" button is absent or hasNext returns false.

Success Criteria

result count >= 1 and username non-null rate = 100%

Known Limitations

• Tab section lazy-loading: The reviews section (along with all tabs: specs, images, recommendations) is lazy-loaded and requires scrolling past the main product area to appear. In some browser sessions or rendering environments, the tabs section does not load even after multiple scroll attempts. This is an intermittent behavior of the Taobao product page rendering engine and does not indicate a site change. Workaround: close and reopen the browser session, then navigate fresh.
• Requires Taobao login; unauthenticated sessions redirect to login page
• Review content is only visible on the product page; there is no standalone reviews URL for Taobao/Tmall products
• Only shows positive buyer reviews by default; negative reviews may require clicking a filter tab within the reviews section (if visible)

Execution Efficiency

• Batch orchestration: Write a bash script to loop through itemIds serially within a single session; add 3–5 second intervals to allow the lazy-loaded reviews section to render.
• Test before batch execution: After writing a batch script, you must first test with 1–2 items to verify the reviews section loads correctly; only then run the full batch. Never skip testing and execute in batch directly.
• Reduce redundant pre-operations: When collecting multiple pages of reviews for one product, stay on the same page and paginate via button click rather than re-navigating.
• Error resumption: Save results page by page; on failure, resume from the last successful page.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/taobao-product-reviews.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Taobao — Shop Catalog

shopId → paginated shop product listing (itemId, title, image URL)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Navigate to a Taobao or Tmall shop's catalog page and extract product listings.

Prerequisites

• Target page is already open in the browser: https://shop{shopId}.taobao.com/category.htm
• User is logged in to Taobao (user avatar or nickname visible in the page header)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Taobao has been confirmed in the current session → skip this step.

Otherwise: open https://www.taobao.com and observe the page header:

• User nickname visible → logged in, continue execution
• Login button visible → not logged in, inform the user that Taobao login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

DOM: shop catalog product list (data extraction)

Navigate to the shop's catalog page, then extract:

1. navigate "https://shop{shopId}.taobao.com/category.htm?search=y&pageNo={page}"
   • Note: Tmall shops redirect to https://{shopName}.tmall.com/category.htm?search=y&pageNo={page}
   • The search=y parameter activates the paginated search mode
2. wait stable
3. eval "$(python scripts/extract-catalog.py '{shopId}' --page {page})"

Parameters:

• shopId: numerical shop ID (e.g., 67095450); found in shop URL as shop{shopId}.taobao.com
• --page: page number, 1-based, default 1

Output example:

[
  {
    "itemId": "1041516493508",
    "title": "绿联T8梯形排插插座转换器插线板大间距宿舍桌面充电多孔位插排",
    "imageUrl": "https://img.alicdn.com/imgextra/...",
    "itemUrl": "https://detail.tmall.com/item.htm?id=1041516493508"
  }
]

Notes:

• price is not included — Taobao shop catalog pages use font-based price obfuscation that cannot be decoded via DOM extraction. Use the taobao-product-detail skill to fetch prices for specific items.
• imageUrl may be a lazy-loaded URL from data-ks-lazyload-custom attribute when the image has not scrolled into view

Error handling: if result count = 0, check that the page loaded correctly (screenshot), confirm the shopId is valid, and retry. Some shops may be Tmall-only and require following the redirect URL.

Pagination

URL Pagination: URL pattern https://shop{shopId}.taobao.com/category.htm?search=y&pageNo={N} (or https://{shopName}.tmall.com/category.htm?search=y&pageNo={N} after redirect), increment pageNo from 1. Each page returns up to 60 items. Termination: when result count = 0 or next page link href with pageNo={N+1} is absent from the DOM.

Next page link selector: a[href*="pageNo"] (contains the next page number).

Success Criteria

result count >= 1 and itemId non-null rate = 100%

Known Limitations

• Prices are font-obfuscated on the shop catalog page and cannot be extracted; fetch individual item prices via taobao-product-detail
• Some shop categories are not shown in the default category.htm view; category-specific browsing requires clicking category links in the shop nav
• Shop redirect from shop{id}.taobao.com to {name}.tmall.com changes the URL structure; the script handles both

Execution Efficiency

• Batch orchestration: Write a bash script to loop through pages serially; add 2–3 second intervals between navigations.
• Test before batch execution: After writing a batch script, you must first test with 1–2 pages to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly.
• Reduce redundant pre-operations: Stay in the same session for all pages of one shop without re-login checks.
• Error resumption: Save results page by page during batch processing; on failure, resume from the last successful page rather than starting over.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/taobao-shop-catalog.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Web Research Assistant

This skill leverages BrowserAct API to provide powerful web research capabilities. When primary web access is restricted or blocked, it automatically searches the internet to find and extract relevant information, ensuring your research tasks are completed successfully.

✨ OpenClaw & Claude Code Optimized

🚀 Works Seamlessly with OpenClaw and Claude Code

Why This Skill?

• 🎯 Purpose-built for OpenClaw and Claude Code
• 🔄 Auto-recovery when web access is restricted
• 🌐 Global access - no IP or geoblocking issues
• 💰 Cost-effective - saves token usage
• ⚡ Fast execution - BrowserAct powered

🎯 When to Use

Web Access Restricted? No Problem!

Use this skill when:

• 🔒 Target websites block AI assistant access
• 🌍 Geographic restrictions limit content access
• 🔐 Paywalls prevent data extraction
• ⏱️ Need quick supplemental information
• 📊 Research requires multiple data sources
• 🔍 Need current information beyond cached data

🔑 API Key Guidance

Before running, check the BROWSERACT_API_KEY environment variable. If not set, request the API key from the user:

Required Message:

"Please provide your BrowserAct API Key from BrowserAct Console to enable web research capabilities."

🛠️ Input Parameters

Configure research based on your needs:

💻 Quick Start

# Basic research
python web-research-assistant/scripts/research.py "AI technology trends"

# Deep research with more results
python web-research-assistant/scripts/research.py "competitor analysis" --max-results 20

# Current news research
python web-research-assistant/scripts/research.py "market trends" --content-type news

# Save to file
python web-research-assistant/scripts/research.py "industry data" -o research_report.md

📊 Output

Structured Research Data:

• Titles, URLs, snippets, and relevance scores
• Key facts and statistics extraction
• Source citations and references
• Comprehensive research report

🎪 What This Skill Does

1. 🔄 Auto-Supplement - When direct access fails, automatically searches the web
2. 🌐 Global Search - Bypasses geographic restrictions via BrowserAct
3. 📈 Multi-Source - Aggregates data from multiple search results
4. ✅ Validation - Cross-references information for accuracy
5. 📋 Report Generation - Creates comprehensive research reports

Features

1. No hallucinations, ensuring stable and accurate data extraction: Pre-set workflows eliminate AI-generated hallucinations.

2. No human verification challenges: Built-in bypass mechanisms eliminate the need to handle reCAPTCHA or other verification challenges.

3. No IP access restrictions or geofencing: Overcomes regional IP limitations for stable global access.

4. Faster execution speed: Tasks complete more rapidly than purely AI-driven browser automation solutions.

5. Exceptional cost-effectiveness: Significantly reduces data acquisition costs compared to token-intensive AI solutions.

🔧 BrowserAct API Integration

Research Request → BrowserAct Search → Data Extraction → Validation → Final Report

Powered by BrowserAct MCP for reliable, unrestricted web access.

⚠️ Error Handling

• Invalid API Key: Report to user, do not retry
• Search Failed: Automatic retry with fallback engine
• No Results: Return partial data with notification
• Timeout: Extended timeout for large research tasks

📖 Best Practices

1. Use specific queries for targeted results
2. Apply time filters for current information
3. Cross-reference key findings
4. Choose appropriate content types for research goals
5. Validate statistics with multiple sources

🔗 Related Skills

• amazon-competitor-analyzer - Amazon competitive intelligence
• google-maps-search-api-skill - Business data extraction
• google-news-api-skill - News monitoring

Version: 1.0.0
Updated: 2026-03-27
API: BrowserAct MCP

Instagram — Place Posts

location name/ID → paginated list of posts tagged at that location

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Search for a location by name, then fetch posts tagged at that location using the internal locations sections API with cursor-based pagination.

Prerequisites

• Browser is open on any Instagram page: https://www.instagram.com/
• Logged into Instagram (user avatar or username visible in top-right corner)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Instagram has been confirmed in the current session → skip this step.

Otherwise: open https://www.instagram.com/ and observe the page login status:

• Logout/sign-out entry, user avatar, or username exists → logged in, continue execution
• Login/register entry exists with no logout entry → not logged in, inform the user that login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: search for location ID

eval "$(python scripts/search-location.py '{keyword}')"

Parameters:

• keyword: Location name to search for (e.g., New York, Eiffel Tower, Times Square)

Output example:

{
  "venues": [
    {
      "id": "212988663",
      "name": "New York, New York",
      "address": "New York, NY",
      "lat": 40.7128,
      "lng": -74.0059
    }
  ]
}

API: get posts for a location

eval "$(python scripts/get-place-posts.py '{location_id}' --tab ranked --max-id '{cursor}' --session-id '{session_id}')"

Parameters:

• location_id: Numeric location ID (from search-location output)
• --tab: Feed type, ranked (top posts, default) or recent (chronological)
• --max-id: Pagination cursor; leave empty for first page, use next_max_id from previous response for subsequent pages
• --session-id: Session identifier for deduplication across pages; use a consistent value per scraping session (e.g., session-001)

Output example:

{
  "items": [
    {
      "pk": "3904677093853259503",
      "code": "DYwMyEFumbv",
      "media_type": 2,
      "taken_at": 1779686199,
      "like_count": 1203,
      "comment_count": 48,
      "caption": "NYC skyline at sunset 🌆",
      "thumbnail_url": "https://scontent.cdninstagram.com/...",
      "video_url": "https://scontent.cdninstagram.com/...",
      "username": "nyc_photos",
      "user_id": "330873185",
      "location_name": "New York, New York"
    }
  ],
  "more_available": true,
  "next_max_id": "32120594f3584430b0d3a4e72372c376"
}

Composite: fetch all posts for a location by name

1. navigate https://www.instagram.com/ → wait stable
2. eval "$(python scripts/search-location.py '{location_name}')" → select the best-matching venue, extract its id as location_id
3. eval "$(python scripts/get-place-posts.py '{location_id}' --session-id 'session-001')" → collect items, note more_available and next_max_id
4. While more_available is true: a. eval "$(python scripts/get-place-posts.py '{location_id}' --max-id '{next_max_id}' --session-id 'session-001')" → accumulate items
5. Merge all collected items

Pagination

API Pagination: max_id, type: cursor, start value: empty string. Next page value source: next_max_id field in response. Termination: more_available is false or next_max_id is null. Keep session_id consistent across all pages for the same scraping session.

Success Criteria

result count >= 1 AND items[0].pk non-null AND items[0].location_name non-null

Known Limitations

• Login required: location sections API requires authentication (CSRF token from logged-in session)
• Location search may return multiple venues with similar names; select the best match by name and address
• Posts count per page varies (approximately 30 posts) depending on media type distribution in sections

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/instagram-scraper-instagram-place-posts.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Instagram — Post Comments

post shortcode or media ID → paginated list of comments

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Fetch comments for a specific Instagram post using the internal comments API with cursor-based pagination.

Prerequisites

• Browser is open on any Instagram page: https://www.instagram.com/
• Logged into Instagram (user avatar or username visible in top-right corner)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Instagram has been confirmed in the current session → skip this step.

Otherwise: open https://www.instagram.com/ and observe the page login status:

• Logout/sign-out entry, user avatar, or username exists → logged in, continue execution
• Login/register entry exists with no logout entry → not logged in, inform the user that login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: get media ID from shortcode

eval "$(python scripts/get-media-id.py '{shortcode}')"

Parameters:

• shortcode: Post shortcode from the Instagram URL (e.g., from instagram.com/p/BwrsO1Bho2N/ → BwrsO1Bho2N)

Output example:

{
  "media_id": "3900557621921709539",
  "shortcode": "BwrsO1Bho2N",
  "media_type": 1,
  "username": "natgeo",
  "taken_at": 1779686199
}

API: get comments page

eval "$(python scripts/get-post-comments.py '{media_id}' --min-id '{cursor}')"

Parameters:

• media_id: Numeric media ID / pk (from get-media-id output, or from profile-posts output)
• --min-id: Pagination cursor; leave empty for first page, use next_min_id from previous response for subsequent pages

Output example:

{
  "comments": [
    {
      "pk": "17857015000000001",
      "text": "Amazing photo!",
      "username": "john_doe",
      "user_id": "123456789",
      "created_at": 1779686500,
      "like_count": 42,
      "reply_count": 3
    }
  ],
  "has_more_comments": true,
  "next_min_id": "17857015000000001"
}

Composite: fetch all comments for a post

1. navigate https://www.instagram.com/ → wait stable
2. If only shortcode is available: eval "$(python scripts/get-media-id.py '{shortcode}')" → extract media_id
3. eval "$(python scripts/get-post-comments.py '{media_id}')" → collect comments, note has_more_comments and next_min_id
4. While has_more_comments is true: a. eval "$(python scripts/get-post-comments.py '{media_id}' --min-id '{next_min_id}')" → accumulate comments
5. Merge all collected comments

Pagination

API Pagination: min_id, type: cursor, start value: empty string. Next page value source: next_min_id field in response. Termination: has_more_comments is false or next_min_id is null.

Success Criteria

result count >= 1 AND comments[0].text non-null AND comments[0].username non-null

Known Limitations

• Login required: comments API returns require_login: true without authentication
• Posts with comments disabled return empty comments array
• Deleted comments or accounts may return partial data

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/instagram-scraper-instagram-post-comments.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Instagram — Profile Metadata

username → profile details (bio, counts, verification status)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Fetch complete metadata for an Instagram user profile via the internal REST API.

Prerequisites

• Browser is open on any Instagram page: https://www.instagram.com/
• No login required for this capability

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: get profile metadata

eval "$(python scripts/get-profile-meta.py '{username}')"

Parameters:

• username: Instagram username without @ (e.g., natgeo)

Output example:

{
  "id": "787132",
  "username": "natgeo",
  "full_name": "National Geographic",
  "biography": "The official Instagram page of National Geographic magazine.",
  "follower_count": 283000000,
  "following_count": 163,
  "media_count": 29500,
  "is_verified": true,
  "is_business_account": true,
  "profile_pic_url": "https://scontent.cdninstagram.com/...",
  "external_url": "https://www.nationalgeographic.com",
  "is_private": false
}

Pagination

Not applicable — single user metadata.

Success Criteria

result count >= 1 AND id field non-null AND username field matches input

Known Limitations

• Private accounts: metadata (follower_count, biography) is returned, but posts are not accessible without following
• Rate limiting may occur with high-frequency requests

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/instagram-scraper-instagram-profile-meta.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Instagram — Profile Posts

username → paginated list of posts (media, caption, counts, timestamps)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Fetch all posts from an Instagram user's profile using the internal feed API with cursor-based pagination.

Prerequisites

• Browser is open on any Instagram page: https://www.instagram.com/
• Logged into Instagram (user avatar or username visible in top-right corner)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Instagram has been confirmed in the current session → skip this step.

Otherwise: open https://www.instagram.com/ and observe the page login status:

• Logout/sign-out entry, user avatar, or username exists → logged in, continue execution
• Login/register entry exists with no logout entry → not logged in, inform the user that login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: get user ID from username

eval "$(python scripts/get-user-id.py '{username}')"

Parameters:

• username: Instagram username without @ (e.g., natgeo)

Output example:

{
  "id": "787132",
  "username": "natgeo",
  "is_private": false
}

API: get profile posts page

eval "$(python scripts/get-profile-posts.py '{user_id}' --count 12 --max-id '{cursor}')"

Parameters:

• user_id: Numeric user ID (from get-user-id output)
• --count: Posts per page, default 12 (max 12)
• --max-id: Pagination cursor; leave empty for first page, use next_max_id from previous response for subsequent pages

Output example:

{
  "items": [
    {
      "pk": "3900557621921709539",
      "code": "DYwOA07iPmB",
      "taken_at": 1779686199,
      "media_type": 1,
      "like_count": 45230,
      "comment_count": 312,
      "caption": "Caption text here...",
      "thumbnail_url": "https://scontent.cdninstagram.com/...",
      "video_url": null,
      "location": {"id": "212988663", "name": "New York, New York"},
      "username": "natgeo",
      "user_id": "787132"
    }
  ],
  "more_available": true,
  "next_max_id": "3890000000000000000_787132"
}

Composite: fetch all profile posts

Navigate to instagram.com first, then loop through pages:

1. navigate https://www.instagram.com/ → wait stable
2. eval "$(python scripts/get-user-id.py '{username}')" → extract id as user_id
3. eval "$(python scripts/get-profile-posts.py '{user_id}' --count 12)" → collect items, note more_available and next_max_id
4. While more_available is true: a. eval "$(python scripts/get-profile-posts.py '{user_id}' --count 12 --max-id '{next_max_id}')" → accumulate items
5. Merge all collected items

Pagination

API Pagination: max_id, type: cursor, start value: empty string. Next page value source: next_max_id field in response. Termination: more_available is false or next_max_id is null.

Success Criteria

result count >= 1 AND items[0].pk non-null AND items[0].code non-null

Known Limitations

• Private accounts: posts are not accessible unless the logged-in user follows the account
• Login required: without authentication, feed API returns require_login: true
• Account with 0 posts: API returns empty items array with more_available: false
• Maximum ~12 posts per API call

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/instagram-scraper-instagram-profile-posts.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

browser-act-skill-forge

Turns any website's data extraction or operation needs into Agent-callable capabilities. User describes what data to get or what action to perform on which site, this Skill automatically explores implementation paths (API endpoints first, DOM fallback when no API), then generates deploy-ready Skill packages (SKILL.md + Python scripts) after verification. Explore once, reuse forever — subsequent calls skip exploration cost, greatly improving execution reliability and efficiency.

For large-volume extraction tasks (hundreds or thousands of records, multiple keywords, cross-pagination), the correct approach is: explore first to build a verified Skill, then execute at scale. Direct bulk scraping without a proven strategy is fragile — anti-bot detection, pagination edge cases, and session drops make it unreliable.

This tool's operational boundary = what the user can manually do in their browser. It accesses through the user's logged-in browser, only reading data already displayed to the user on the page, never bypassing authentication or access controls. Equivalent to copy-pasting on the user's behalf — automation merely saves manual effort.

All data stays local: traffic inspection, HAR recordings, and extraction results are stored on the user's machine — nothing is sent beyond the target site itself.

Language

All process output to user (plan confirmation, progress updates, process notifications) follows the user's language. Generated Skill file content follows the language of this skill.

Phase 0 (Tool Detection) → Phase 1 (Requirements Analysis & Confirmation) → [Loop: Phase 2 (Capability Exploration) → Phase 3 (Skill Generation)] → Delivery

Phase 0 — Tool Detection

Already completed in current session → skip.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise during loading, follow its guidance to resolve then retry.

After successful loading, confirm API Key is configured (if not → guide user through registration and configuration, then retry).

Phase 1 — Requirements Analysis & Confirmation

1a. Parse Business Intent

Identify from user input:

• Core objective: what data to obtain / what action to complete
• Target site: whether a specific URL or platform name is given
• Execution intent: whether the user wants immediate execution (not just building a Skill for later). Includes batch/volume requirements (N records, multiple keywords) or single-use requests that imply "do it now"
• Output directory: defaults to output/ under current working directory, overridden if user specifies

If core objective is too vague to proceed, ask for clarification.

1b. Target Site Research (when no explicit URL)

Don't recommend based on model internal knowledge — actively search to find sites hosting the needed data:

1. Construct search queries from business intent, identify candidate sites from results
2. Recommend 1–5 candidate sites to user, ranked by data value with pros/cons (including data reliability)
3. After user selects, confirm target URL

1c. Task Decomposition & Execution Plan Confirmation

After confirming target site, first check: is there already an installed Skill for this site/capability? If yes → inform user and skip to Delivery step 4 (batch execution).

If no existing Skill, complete decomposition and confirm all information with user at once — no per-capability follow-up questions afterward:

1. Identify independent stages involved (search, list page, detail page, login, submission…)
2. Determine type: extraction (get data) vs operation (perform action)
3. Splitting criteria: If you swap the business objective, can this stage be reused independently? Yes = independent capability. Cross-page steps serving the same business objective (e.g., list page collection + detail page extraction) stay as one capability, orchestrated via composite components
4. Set skill-name and capability directory names (lowercase English, hyphen-separated), create directories under output/{skill-name}/ (use user-specified path if given)
5. Confirm complete execution plan with user:

Target site: {url}
Output: output/{skill-name}/

Capabilities (executed in order):
1. {site-slug}-{capability-slug} ({extraction/operation}) — {one-line description}
2. {site-slug}-{capability-slug} ({extraction/operation}) — {one-line description}
...

If execution intent was identified in 1a, append to the plan:

Pipeline:
1. Explore site → discover and verify viable API endpoints or DOM extraction methods
2. Generate Skill files (SKILL.md + scripts)
3. Automated testing to confirm Skill works
4. Install Skill
5. Read installed Skill → write and run batch scripts to fulfill user's original task

Present the plan and wait for user to confirm or adjust. Do not ask separate questions about items that have reasonable defaults (output directory, naming conventions, etc.).

After user confirms, enter execution loop with no mid-process questions.

Phase 2 and Phase 3 below execute in a loop for each capability unit — complete one before starting the next.

Phase 2 — Capability Exploration

Read the corresponding reference file based on capability type:

• Extraction → references/exploration_extraction.md
• Operation → references/exploration_operation.md

Goal: prioritize API endpoints for target capability; fall back to DOM operations when API isn't viable. Record complete reproducible invocation methods.

Success criteria:

• Can stably obtain target data / trigger target action (API or DOM path)
• Complete invocation/operation method recorded (endpoint + params, or selectors + interaction steps)
• Enum parameters collected for all meaningful values

When a means fails, follow this sequence:

1. Do not retry with different parameters (varying parameters rarely changes the outcome)
2. Return to the goal itself
3. Enumerate all alternative means that could achieve the goal
4. Pick the next one and execute

A deterministic failure (explicit error code, structural mismatch) confirms the means is unviable in one attempt. A transient failure (timeout, connection drop) warrants one retry — but not more.

Exploration cap: 100 tool call steps. If still unable to progress, report known obstacles to user and ask for next steps.

Don't touch experience notes: experience notes (browser-act-skill-forge-memories/) are for generated Skills' future Agent use — neither read nor write during exploration and generation phases.

Phase 3 — Skill Generation

Read references/output_template.md for file format specification.

3a. JS Encapsulation

Encapsulate each verified JS snippet from exploration into an independent Python file:

1. Identify business parameters (keywords, page number, sort order, etc.) → extract as argparse arguments
2. Hardcode selectors, field mappings, endpoint URLs as fixed values in JS f-string
3. Escape JS curly braces as {{ }} (f-string syntax requirement, otherwise Python errors)
4. Write to scripts/{feature-name}.py

3b. Encapsulation Verification

Run end-to-end verification for each .py file:

1. python scripts/{feature-name}.py {test-params} — confirm output is valid JS string
2. eval "$(python scripts/{feature-name}.py {test-params})" — confirm browser execution result matches exploration phase
3. Simulate error scenarios (e.g., non-existent ID, navigating to wrong page), confirm returns {"error": true, "message": "..."} rather than crashing

Verification failure → fix .py file and retry, never skip.

3c. Generate SKILL.md

Create SKILL.md per template, capability component section references scripts/*.py invocation commands (no inline JS).

Output directory structure:

output/{skill-name}/{site-slug}-{capability-slug}/
├── SKILL.md
└── scripts/
    └── {feature-name}.py

After generation, briefly inform user: capability name, output path, primary implementation approach (API / Network capture / DOM / hybrid).

3d. Compliance Self-Check

Two checks — must Read generated files and execute verification commands as evidence; mental assertion alone does not count:

1. Process: Re-read the exploration reference file used in Phase 2 and the output steps above (3a–3c), confirm each defined step was actually executed, not skipped
2. Output: Read generated scripts/*.py and SKILL.md, check against the Filling Specifications in output_template.md and the Code / JS Execution Environment / DOM Operation constraints defined earlier in this skill

Any gap found → go back, complete the missing step or fix the output, then re-verify.

Delivery Flow

After all capabilities are generated, proceed in this order:

1. Automated Testing

Start testing immediately after generation — no user confirmation needed. Auto-design minimal test cases based on generated capability components — use fewest inputs to cover all functional paths (each atomic component called at least once, composite components run full flow).

Must execute testing via Sub-Agent — do not test directly in the main session. Dispatch the following prompt:

Read {absolute path to SKILL.md} as your execution guide.

Test cases:
{auto-generated test case list, each annotated with which component it covers}

Execution requirements:
- Follow SKILL.md instructions strictly, don't use methods outside the guide
- Record specific issues if SKILL.md instructions are unclear and prevent progress

Report after execution:
1. Execution result per component (pass/fail)
2. Failure reasons (if any)
3. Unclear parts in SKILL.md instructions (if any)
4. Severe accuracy or performance issues (don't report non-severe)
5. Output data summary

Test failure → fix Skill and retest until passing.

2. Install Skill

Install the generated Skill from the output directory. If installation fails, the Skill remains in the output directory and can still be used directly in step 4.

3. Report Results

After tests pass, report to user:

• Generated Skill list (name + path + contained files)
• Data coverage (fields + status, don't list data source or implementation method)
• Incomplete coverage gaps (failed enum parameters, missing target fields, uncovered filter conditions, etc.)
• Test results summary

4. Execute (if execution intent was identified in Phase 1)

If execution intent was identified in Phase 1:

1. Invoke the installed Skill via the Skill tool to read its full content. If installation failed in step 2, read the SKILL.md directly from the output directory instead
2. Follow the Skill's instructions to execute the user's original task in the current session
3. For batch/volume tasks, write batch execution scripts according to the Skill's guidance

If no execution intent was identified (user only wanted to build a Skill for later use), end here.

Tool Constraints

Phase 2 (Capability Exploration), Phase 3 (Skill Generation), and Delivery testing must follow these rules.

File Management

All intermediate artifacts (HAR files, temp records, debug output) go in the tmp/ directory. Create it first if it doesn't exist.

browser-act

• Network data is page-scoped — must re-wait and re-read after navigating to a new page
• Wait for network stability before reading traffic: whether triggered by page navigation or UI interaction, use wait stable before reading network requests
• Wait for elements before operating on async DOM: for async-injected content (browser extensions, lazy-loaded components), use wait --selector "{target selector}" --state attached --timeout {ms} before interacting
• No JS-level network interception: never override XMLHttpRequest.prototype, window.fetch, etc. Use network requests / network request <id> for endpoint discovery
• network clear only before navigation/reload: clearing traffic loses all observed request records. Use --filter for routine filtering, not clear. To track requests from specific interactions, use network har start → interact → network har stop instead of clear + re-read

DOM Operation Constraints

Applies to all DOM operation scenarios (data extraction, enum collection, pagination controls, form submission, API field supplementation):

Selector priority: data-testid > id > name > aria-label > structural path. Avoid pure positional indexes (:nth-child / [1]) unless structure is genuinely stable.

Batch-validate selectors: test all candidate selectors in a single eval call, return JSON summary (hit count per selector, key attributes of first element, uniqueness). Never eval selectors one by one — each eval is a browser roundtrip.

Shadow DOM: when target element is inside a Shadow Root, access via element.shadowRoot.querySelector, split selector into two parts (host element + Shadow-internal path).

Three-layer selector validation: element assertion (expected attributes match) → result check (non-empty, reasonable count) → success criteria. Must be tested on the real page, never written speculatively from DOM structure.

Control scan (during enum collection): use one eval to return complete mapping of all target controls (tag+type / name+id / placeholder / label). Traverse up from control to find nearest form item container for label text; don't hardcode component library class names; component libraries associate labels with inputs via DOM hierarchy nesting, not label[for="xxx"].

state index dynamic allocation: state returns element indexes that are dynamically allocated per session — never write them into strategy code, only use them at execution time in real-time.

Code Constraints

Must directly operate on target site: never obtain data through external services (including third-party scraping platforms, data aggregation APIs, proxy services), and never call the target site's official open platform API (rationale: generated Skills target zero-config deployment without requiring users to register developer API keys or manage credentials). Solutions must access the target site directly through the browser, using its frontend's internal endpoints or DOM data — the same resources already visible to the authenticated user.

Framework internal state fast-fail: when attempting to access page data or element info through framework internals (__vue_app__, $data, React fiber, Angular ng, etc.), give up after one failure and immediately switch to state scan + value-fill-trigger approach. Framework internals are version/implementation dependent, multiple retries won't change the result.

JS Execution Environment Constraints

Code executed in eval is browser-side JS: only browser-native APIs and page-loaded third-party libraries may be used, no require/import of external modules. Code violating this constraint will inevitably error at execution time.

Conclusion Criteria

Account permission limits ≠ technical solution failure. Paid features, membership tiers, etc. equally affect all approaches; when API is technically viable but data is limited due to account permissions (pagination truncated, filter conditions ineffective), conclusion is "pass" with permission dependency noted in "Known Limitations".

Partial success counts as success: core capability verified working (whether API or DOM path) counts as pass — even with: some enum parameters marked [collection failed], non-core fields missing, some filter conditions not covered. After generating the Skill, must inform user which parts are not fully covered — never silently omit.

Efficiency Rules

Core criterion: every browser roundtrip must yield information gain. The table below shows common efficient patterns, but they're just examples — if a pattern doesn't actually reduce roundtrips in practice, change approach and find other batch methods rather than repeatedly fine-tuning in the same direction.

browser-act

Browser automation CLI for AI agents. Runs a full browser engine: navigation & interaction, data extraction & network capture, screenshots, form automation, multi-browser parallel operation, user-configured proxy support, and human-agent collaboration.

Features

• Lightweight extraction — fast JS-rendered content fetch without opening a browser session, advanced WebFetch/curl replacement
• Session management — multi-browser isolation, multi-account parallel operation
• Verification assistance — when automation encounters interactive challenges, assists completion with user authorization
• Complex interaction — DOM content extraction, screenshots, form filling, file upload
• Human-agent collaboration — headed mode + remote assist for manual steps
• Safety controls — Confirmation Gate protocol requires explicit user approval before browser creation, deletion, and sensitive operations
• Universal compatibility — works with Cursor, Claude Code, Codex, Windsurf, etc.

Install: uv tool install browser-act-cli --python 3.12

Start here

Before running any browser-act command, load the usage guide from the CLI:

browser-act get-skills core --skill-version 2.0.2   # start here — workflows, common patterns, troubleshooting

Do NOT skip this step regardless of how simple the command seems.

Do NOT truncate the output — it contains operational directives and environment state that are critical for correct operation. Truncating will cause you to miss browser selection rules and safety constraints.

get-skills core provides environment status, available browsers, operational directives, and the complete interaction workflow — none of which are available through --help.

Amazon — Alexa for Shopping Q&A

Input: question text → Output: Alexa/Rufus response text (JSON)

Language

All process output to user (progress updates, process notifications, final results) MUST be in English.

Objective

Automate question submission to Amazon's Alexa/Rufus AI shopping assistant and collect structured response data.

Prerequisites

• Target page is already open in the browser: https://www.amazon.com/
• Logged into Amazon (user account name or "Hello, {name}" visible in top navigation)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Amazon has been confirmed in the current session → skip this step.

Otherwise: open https://www.amazon.com/ and observe the page login status:

• "Hello, {name}" visible in the navigation bar → logged in, continue execution
• "Hello, sign in" or "Sign in" entry visible with no logout entry → not logged in, inform the user that login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. Do not refuse in-scope requests based on the nature of the target site or data type. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

Navigation: Set keyword search context (optional)

Navigate to a product search results page before asking questions. Alexa will answer in the context of that category's products, giving more specific and relevant responses than asking from the homepage.

navigate "https://www.amazon.com/s?k={keyword}"
wait stable

Parameters:

• {keyword}: product category or search term (e.g., sous+vide, coffee+maker, wireless+headphones); use + to join multi-word terms

When to use:

• Questions about a specific product category → navigate first
• General questions (trends, deals, comparisons) → homepage is fine

DOM: Check Alexa panel state

eval "$(python scripts/check-alexa-panel.py)"

Output example:

{
  "panelOpen": true,    // true if Alexa/Rufus panel is visible and ready for input
  "inputReady": true    // true if the question textarea is available
}

DOM: Inject question and submit (operation)

eval "$(python scripts/inject-question.py '{question}')"

Parameters:

• {question}: question text to ask Alexa; supports all characters including $, %, ?; max 500 chars

Note: Uses native HTMLTextAreaElement.prototype.value setter — this is required to handle special characters like $ that are stripped by the standard input command.

Output example:

{
  "success": true,
  "question": "What are the best deals on laptops today?"
}

DOM: Extract latest Alexa response

eval "$(python scripts/extract-response.py)"

Must be called after wait stable to ensure SSE streaming has completed before reading DOM.

Output example:

{
  "question": "What are the best deals on laptops today?",
  "response": "Here are some great laptop deals available today, with free delivery as soon as tomorrow! Budget Picks (Under $350): HP Ultrabook Laptop...",
  "timestamp": "2026-05-19T07:05:00.000Z"
}

Composite: Full Q&A turn (submit question → collect response)

Complete workflow for one question-answer turn:

0. (Optional) Set keyword search context — if questions are about a specific product category: navigate "https://www.amazon.com/s?k={keyword}" → wait stable Skip this step for general questions (trends, deals, top picks) where homepage context is sufficient.
1. eval "$(python scripts/check-alexa-panel.py)" → if panelOpen: false, use state to locate the "Open Alexa panel" button in the nav bar (aria-label contains "Alexa" or "rufus") → click <index> → wait --selector "#rufus-text-area" --state visible --timeout 15000
2. eval "$(python scripts/inject-question.py '{question}')" → confirm success: true
3. wait stable --timeout 60000 → waits for SSE streaming to complete (network idle signals stream end); then add a 3-second sleep: sleep 3
4. eval "$(python scripts/extract-response.py)" → collect {question, response, timestamp}

Error handling:

• If inject-question.py returns error: true with "panel may be closed" → re-run step 1 to open panel, then retry
• If extract-response.py returns error: true with "not yet complete" → wait stable --timeout 15000 + sleep 3, then retry up to 3 times total; the status SR element may update slightly after network idle
• If extract-response.py returns error: true with "status element not found" → panel may have closed; re-run step 1

Batch questions example — with keyword search context (bash loop):

# Navigate to category page once, then ask all related questions
SESSION="amazon-qa"
KEYWORD="sous+vide"
SKILL_DIR=".claude/skills/amazon-alexa-qa"

browser-act --session $SESSION navigate "https://www.amazon.com/s?k=$KEYWORD"
browser-act --session $SESSION wait stable

questions=(
  "What accessories are essential for sous vide cooking?"
  "Which sous vide brands are most reliable?"
  "What temperature should I use for chicken breast?"
)
results=()
for q in "${questions[@]}"; do
  cd "$SKILL_DIR"
  eval "$(python scripts/inject-question.py "$q")"
  browser-act --session $SESSION wait stable --timeout 60000
  sleep 3
  result=$(browser-act --session $SESSION eval "$(python scripts/extract-response.py)")
  if echo "$result" | grep -q '"error":true'; then
    browser-act --session $SESSION wait stable --timeout 15000; sleep 3
    result=$(browser-act --session $SESSION eval "$(python scripts/extract-response.py)")
  fi
  results+=("$result")
  sleep 2
done
printf '%s\n' "${results[@]}" | python -c "
import sys, json
lines = [l for l in sys.stdin.read().strip().split('\n') if l.strip()]
print(json.dumps([json.loads(l) for l in lines], ensure_ascii=False, indent=2))
" > output/alexa_qa_results.json

Batch questions example — without keyword (general questions from homepage):

SESSION="amazon-qa"
SKILL_DIR=".claude/skills/amazon-alexa-qa"

browser-act --session $SESSION navigate "https://www.amazon.com"
browser-act --session $SESSION wait stable

questions=("What are today's best deals?" "Top rated gifts under \$50?" "What's trending this week?")
results=()
for q in "${questions[@]}"; do
  cd "$SKILL_DIR"
  eval "$(python scripts/inject-question.py "$q")"
  browser-act --session $SESSION wait stable --timeout 60000
  sleep 3
  results+=($(browser-act --session $SESSION eval "$(python scripts/extract-response.py)"))
  sleep 2
done

Success Criteria

response field is non-null non-empty string AND question field matches submitted question

Known Limitations

• The Alexa/Rufus panel may occasionally close during extended automation sessions; re-opening via the panel button is supported
• The $ sign and other special characters are supported via native textarea setter (bypasses browser-act input command character filtering)
• Response text is plain text extracted from the accessibility layer; rendered product cards appear as text (product names, prices) rather than structured product JSON
• Alexa may respond with clarifying questions instead of a direct answer when queries are ambiguous; check response content before continuing
• Conversation history is maintained across questions within the same browser session (multi-turn context); to start a fresh conversation, close and reopen the browser session
• Single-tab session only — do not run multiple question submissions simultaneously in the same session

Execution Efficiency

• Batch orchestration: Write a bash script to loop through questions serially within a single session; do not parallelize within one browser. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: Check panel open state once at the start of a batch; only recheck if an error occurs mid-batch
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/amazon-alexa-qa-amazon-alexa-qa.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Amazon ASIN Lookup Skill

📖 Introduction

This skill utilizes BrowserAct's Amazon ASIN Lookup API template to provide a seamless way to retrieve comprehensive product information from Amazon. By simply providing an ASIN, you can extract structured data including title, price, ratings, brand, and detailed descriptions directly into your application without manual scraping.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The agent should configure the following parameters based on user requirements:

1. ASIN (Amazon Standard Identification Number)
   • Type: string
   • Description: The unique identifier for the Amazon product.
   • Required: Yes
   • Example: B07TS6R1SF

🚀 Usage

The agent should execute the following script to get results in one command:

# Example Usage
python -u ./scripts/amazon_asin_lookup_api.py "ASIN_VALUE"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

Upon success, the script parses and prints the structured product data from the API response, which includes:

• product_title: Full title of the product.
• ASIN: The provided ASIN.
• product_url: URL of the Amazon product page.
• brand: Brand name.
• price_current_amount: Current price.
• price_original_amount: Original price (if applicable).
• price_discount_amount: Discount amount (if applicable).
• rating_average: Average star rating.
• rating_count: Total number of ratings.
• featured: Badges like "Amazon's Choice".
• color: Color variant (if applicable).
• compatible_devices: List of compatible devices (if applicable).
• product_description: Full product description.
• special_features: Highlighted features.
• style: Style attribute (if applicable).
• material: Material used (if applicable).

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Product Data Enrichment: Retrieve full details for a list of ASINs to update an e-commerce database.
2. Price Comparison: Lookup current Amazon prices for specific ASINs to compare with other retailers.
3. Review Monitoring: Track changes in rating averages and review counts for key products.
4. Availability Checks: Automatically verify if a specific product is currently in stock on Amazon.
5. Brand Analysis: Identify the brand and manufacturer of products identified by ASIN.
6. Detailed Specifications: Fetch material, style, and color information for catalog management.
7. Feature Highlighting: Extract "special features" and detailed descriptions for marketing copy.
8. Compatibility Verification: Check "compatible devices" for electronics or accessories.
9. Market Research: Analyze featured badges like "Amazon's Choice" for specific product IDs.
10. URL Resolution: Convert a list of ASINs into full Amazon product page URLs.

Amazon Best Selling Products Finder API Skill

📖 Skill Introduction

This skill provides users with a one-stop product data extraction service using the BrowserAct Amazon Best Selling Products Finder API template. It can directly extract structured best-selling product data from Amazon. By inputting search keywords, data limit, and marketplace URL, you can easily get clean and usable product data including titles, prices, ratings, reviews, sales volume, and promotional details.

✨ Features

1. No hallucinations, ensuring stable and precise data extraction: Preset workflows avoid AI generative hallucinations.
2. No CAPTCHA issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP access restrictions and geofencing: No need to handle regional IP restrictions.
4. More agile execution speed: Compared to pure AI-driven browser automation solutions, task execution is faster.
5. Extremely high cost-effectiveness: Significantly reduces data acquisition costs compared to AI solutions that consume a large number of Tokens.

🔑 API Key Guide Flow

Before running, first check the BROWSERACT_API_KEY environment variable. If it is not set, do not take other actions; require and wait for the user to collaborate to provide it. The Agent must inform the user at this time:

"Since you have not configured the BrowserAct API Key yet, please go to the BrowserAct Console to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure the following parameters based on user needs:

1. KeyWords

   • Type: string
   • Description: Search keywords used to find Amazon products.
   • Example: iphone 17 pro max, gaming mouse, running shoes

2. Date_limit

   • Type: number
   • Description: Maximum number of products to extract.
   • Default: 10
   • Recommendation: Set to a lower number for quick checks, or higher for comprehensive analysis.

3. Marketplace_url

   • Type: string
   • Description: Amazon marketplace URL for region-specific searches.
   • Default: https://www.amazon.com
   • Example: https://www.amazon.co.uk, https://www.amazon.de

🚀 Call Method (Recommended)

The Agent should execute the following standalone script to achieve "one command to get results":

# Call example
python -u ./scripts/amazon_best_selling_products_finder_api.py "search keywords" limit "marketplace_url"

⏳ Running Status Monitoring

Since this task involves automated browser operations, it may take a long time (several minutes). The script will continuously output status logs with timestamps (e.g. [14:30:05] Task Status: running) during execution. Agent notice:

• While waiting for the script to return the result, please keep an eye on the terminal output.
• As long as the terminal is still outputting new status logs, it means the task is running normally. Please do not mistakenly judge it as a deadlock or unresponsive.
• If the status remains unchanged for a long time or the script stops outputting and no result is returned, the retry mechanism can be triggered.

📊 Data Output Description

After successful execution, the script will parse and print the result directly from the API response. The result contains:

• title: Product title
• brand: Brand name
• list_price: Original list price
• current_price: Current selling price
• star_rating: Average star rating
• review_count: Total review count
• past_month_sales: Sales volume in the past month
• availability: Stock status
• promotion: Promotional offers
• asin: Amazon Standard Identification Number
• category: Product category
• badge: Badges like Amazon's Choice
• product_url: Direct link to the product

⚠️ Error Handling & Retry Mechanism

During the execution of the script, if an error occurs (such as network fluctuation or task failure), the Agent should follow this logic:

1. Check the output content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. At this time, do not retry, and guide the user to recheck and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task execution fails (for example, the output starts with Error: or the returned result is empty), the Agent should automatically try to execute the script again once.

2. Retry limit:

   • Automatic retry is limited to once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Market Research: Extract product listings and ratings to analyze the current market for specific keywords.
2. Competitor Analysis: Monitor competitor pricing, discounts, and sales volume over time.
3. Trending Products Discovery: Find the best-selling and highly rated products within a specific category.
4. Price Monitoring: Track current prices and list prices to optimize purchasing strategies.
5. Cross-Region Analysis: Compare product availability and pricing across different Amazon marketplaces.

Amazon Buy Box Monitor API Skill

📖 Introduction

This skill provides users with an automated Amazon Buy Box monitoring service using the BrowserAct API template. It can directly extract structured data including basic product details, other sellers' prices, and seller ratings from Amazon via a specific ASIN. No coding or proxies are required, and users only need to provide the ASIN and an optional marketplace URL to retrieve clean and usable data.

✨ Features

1. No hallucinations, ensuring stable and accurate data extraction: Pre-set workflows avoid AI generative hallucinations.
2. No CAPTCHA issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP access restrictions or geo-fencing: No need to handle regional IP restrictions.
4. More agile execution speed: Faster task execution compared to purely AI-driven browser automation solutions.
5. Extremely high cost-effectiveness: Significantly reduces data acquisition costs compared to AI solutions that consume massive amounts of tokens.

🔑 API Key Guidance

Before running, you must check the BROWSERACT_API_KEY environment variable. If it is not set, do not take any other actions; you must prompt and wait for the user to provide it. The Agent must inform the user at this time:

"Since you have not configured the BrowserAct API Key, please go to the BrowserAct Console first to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure the following parameters based on the user's needs:

1. ASIN

   • Type: string
   • Description: The Amazon Standard Identification Number. This is the unique identifier for the product on Amazon.
   • Example: B005O2ZU68
   • Required: Yes

2. Marketplace_url

   • Type: string
   • Description: The Amazon marketplace URL indicating the region.
   • Default value: https://amazon.com/
   • Example: https://amazon.co.uk/
   • Required: Yes

🚀 Invocation Method (Recommended)

The Agent should execute the following standalone script to achieve "one command to get results":

# Example invocation
python -u ./scripts/amazon_buy_box_monitor_api.py "ASIN" "Marketplace_url"

⏳ Running Status Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps while running (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script to return results, please keep paying attention to the terminal output.
• As long as the terminal is still outputting new status logs, it means the task is running normally. Do not misjudge it as deadlocked or unresponsive.
• If the status remains unchanged for a long time or the script stops outputting without returning a result, then consider triggering the retry mechanism.

📊 Data Output Description

After successful execution, the script will parse and print the result directly from the API response. The result includes:

• asin: The Amazon Standard Identification Number
• product_title: The title of the product
• buy_box_owner: The owner of the buy box
• buy_box_price: The current buy box price
• currency: The currency of the price
• fulfillment_method: The fulfillment method (e.g., FBA, FBM)
• availability_status: Stock availability status
• other_sellers: An array of other sellers including their name, price, shipping fee, and seller rating
• seller_info: Detailed information about the main seller including rating and feedback count

⚠️ Error Handling & Retry

If an error is encountered during the execution of the script (such as network fluctuations or task failure), the Agent should follow this logic:

1. Check output content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. In this case, do not retry, but guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task fails (e.g., output starts with Error: or returns empty results), the Agent should automatically attempt to execute the script once more.

2. Retry limits:

   • Automatic retry is limited to only once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Query Amazon buy box information: Find out who currently owns the buy box for a specific ASIN.
2. Monitor Amazon product prices: Track the current price and buy box price changes.
3. Extract Amazon product details by ASIN: Get basic product information like title and brand.
4. Check other sellers prices on Amazon: Analyze pricing strategies of competitors for the same product.
5. Get Amazon seller ratings and feedback count: Evaluate the reputation of sellers on the listing.
6. Monitor buy box ownership for a specific ASIN: Check if a particular seller maintains the buy box.
7. Track Amazon fulfillment methods for competitors: Determine whether competitors are using FBA or FBM.
8. Compare Amazon product prices across different sellers: View shipping fees and total prices from multiple sellers.
9. Retrieve Amazon buy box availability status: Check if the product is in stock or backordered.
10. Analyze Amazon seller profile details: Extract detailed seller info and recent feedback summaries.

Amazon Listing Competitor Analysis

📖 Brief

This skill runs a two-phase workflow on a single competitor Amazon listing. Phase 1 uses the BrowserAct Amazon Listing Extractor for SEO template to pull visible product data from that listing. Phase 2 diagnoses what that competitor does well and where the market shows gaps, then closes with your strategic opportunity points (how you can win next to them). Do not end with instructions that read like editing or rewriting this competitor's listing; the analyzed ASIN is evidence only. Final narrative output should be grounded in extracted data, not generic claims.

✨ Features

1. No hallucinations, ensuring stable and accurate data extraction: Pre-set workflows avoid AI generative hallucinations.
2. No CAPTCHA issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP restrictions or geo-blocking: No need to deal with regional IP restrictions or geofencing.
4. Faster execution: Tasks execute faster compared to purely AI-driven browser automation solutions.
5. Extremely high cost-efficiency: Significantly reduces data acquisition costs compared to AI solutions that consume massive amounts of tokens.

🔑 API Key Guide

Before running, you must check the BROWSERACT_API_KEY environment variable. If it is not set, do not take other actions first; you should ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key yet, please go to the BrowserAct Console to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure the following parameters based on user needs:

1. ASIN

   • Type: string
   • Description: The ASIN (Amazon Standard Identification Number) of the Amazon product to analyze.
   • Example: B0CS62LY6P
   • Required: Yes

2. Marketplace_url

   • Type: string
   • Description: The base URL of the Amazon marketplace. Use the correct regional site for the listing.
   • Example: https://www.amazon.com/, https://www.amazon.de/
   • Default: https://www.amazon.com/

🚀 Invocation Method

Run Phase 1 extraction with the script below. After structured data is returned, the Agent performs Phase 2 analysis using the framework in Competitive Analysis Framework (Phase 2). The closing section must synthesize opportunity points for the user's business, not a checklist of edits applied to the competitor page under review.

python -u ./scripts/amazon_listing_competitor_analysis.py "B0CS62LY6P" "https://www.amazon.com/"

When only the ASIN is needed, the marketplace argument may be omitted; the script defaults to https://www.amazon.com/.

⏳ Running Status Monitoring

Since this task involves automated browser operations, it may take a long time (several minutes). The script will continuously output status logs with timestamps while running (e.g., [14:30:05] Task Status: running). Agent guidelines:

• While waiting for the script to return results, please keep an eye on the terminal output.
• As long as the terminal continues to output new status logs, it means the task is running normally. Do not misjudge it as a deadlock or unresponsiveness.
• If the status remains unchanged for a long time or the script stops outputting without returning a result, only then consider triggering the retry mechanism.

📊 Data Output

Upon successful execution, the script prints the API result string (or full task JSON if no string field is present). Typical fields include:

• asin, title, product_url, brand, price, coupon_text, rating, review_count, best_sellers_rank, availability, prime_eligible
• description, short_description, category, key_features, bullet_points
• main_image_url, additional_image_urls, seller_name, ships_from, sold_by
• specifications, product_details, attributes, and review-related blocks (reviewer, content, date, helpful votes, etc.)

Use this payload as the single source of truth for Phase 2; do not invent listing facts.

⚠️ Error Handling & Retry

During script execution, if errors occur (such as network fluctuations or task failure), the Agent should follow this logic:

1. Check the output content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. At this point, do not retry, but guide the user to recheck and provide the correct API Key.
   • If the output contains "concurrent" or "too many running tasks" or similar concurrency limit messages, it means the concurrent task limit for the current subscription plan has been reached. Do not retry; guide the user to upgrade their plan. Agent must inform the user:

     "The current task cannot be executed because your BrowserAct account has reached the limit of concurrent tasks. Please go to the BrowserAct Plan Upgrade Page to upgrade your subscription plan and enjoy more concurrent task benefits."

   • If the output does not contain the above error keywords but the task fails (e.g., output starts with Error: or returns empty results), the Agent should automatically try to run the script once more.

2. Retry limit:

   • Automatic retry is limited to once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Competitor listing teardown: Analyze one ASIN to see title formula, bullets, and differentiation language.
2. Keyword placement audit: Map where primary and long-tail terms appear across title, bullets, and description or A+ content.
3. Visual strategy review: Infer image narrative, infographic highlights, and video approach from extracted media data.
4. Buyer-validated selling points: Use high-helpful positive reviews to confirm what buyers value versus what the listing emphasizes.
5. Unmet needs mining: Use three-star and mixed reviews to find feature and expectation gaps.
6. Pre-launch gap analysis: Compare a planned positioning against a top competitor's listing structure.
7. Cross-marketplace research: Run the same ASIN on different regional Amazon URLs for localized copy signals.
8. Opportunity backlog from a rival listing: Turn extracted facts and gaps into a prioritized map of positioning, search, creative, and offer opportunities for your side of the market.
9. SEO and conversion benchmarking: Relate BSR, rating volume, and copy patterns without guessing unavailable metrics.
10. Review-driven objection handling: Surface recurring complaints to address in copy or images.

🧠 Competitive Analysis Framework (Phase 2)

After extraction succeeds, work through each dimension below. Every insight must be grounded in the actual extracted data.

Layer 1 — What the Competitor Did Right

1. Content Strategy

• Title formula: Information order, primary keyword placement, brand-first vs feature-first vs use-case-first.
• Bullet priority: What Bullet 1 leads with; selling point order across bullets (signal of tested conversion order).
• Differentiation language: How generic category features are phrased to sound distinct.
• A+ content: Modules implied by extracted content (comparison table, brand story, lifestyle, spec callouts).

2. Keyword Placement Strategy

Map where terms appear (not only which terms exist):

• Title (first 80 chars) → primary ranking bets
• Bullets 1–2 → secondary high-weight terms
• Bullets 3–5 → long-tail and use-case terms
• Description / A+ → supplementary terms and synonyms

3. Visual Content Strategy

• Image narrative arc: Sequence story (hero, lifestyle, pain point, specs, size comparison, social proof, guarantee).
• Infographic data: Numbers or attributes highlighted and how they are presented.
• Video (if present in data): Hook length, demo vs lifestyle, subtitles.
• Overall style: Premium, approachable, technical, lifestyle-focused.

4. Buyer-Validated Selling Points

From four- to five-star reviews with high helpful votes:

• What reviewers praise that the listing underplays
• Unexpected benefits buyers mention

Layer 2 — What the Market Lacks

5. Unmet Buyer Needs

From three-star reviews and recurring themes in low stars (non-defect noise):

• "I wish it had…", "Would be five stars if…", "Good but not great because…"

6. Keyword Gaps

• Natural search terms buyers would use that the listing does not cover
• High-traffic angles the data suggests but copy does not foreground

7. Visual Content Gaps

• Weak or missing context in existing images
• Absent image types (use-case, comparison, real-world scale)

Required Output Format (Phase 2)

Produce the analysis using this structure. Be specific and quote or paraphrase extracted fields and reviews where useful. The final block is your opportunity synthesis; avoid imperatives that sound like "change this competitor's bullet five" or any direct edit list for the ASIN being studied.

Competitor ASIN: [ASIN] | Brand: [brand] | BSR: [rank] | Rating: [x.x] ([N] reviews)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

✅ WHAT THIS COMPETITOR DOES RIGHT

Content Strategy:
  - Title formula: [describe the pattern and keyword placement]
  - Bullet priority: [what each bullet leads with and the logic behind the order]
  - Standout phrasing: [specific language worth noting or borrowing]
  - A+ modules: [which are used and what they emphasize]

Keyword Placement:
  - Primary (title, first 80 chars): [keywords]
  - High-weight (Bullets 1–2): [terms]
  - Long-tail (Bullets 3–5): [terms]
  - Supplementary (description/A+): [terms]

Visual Strategy:
  - Image sequence: [describe the narrative arc across images]
  - Infographic highlights: [what data/specs are called out]
  - Video: [approach if present, or "none"]

Buyer-Validated Selling Points:
  - "[specific insight from high-helpful reviews]"
  - "[another insight]"

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🕳️ MARKET GAPS (OBSERVED ON THIS COMPETITOR LISTING)

Content gap: [selling points or use cases their copy under-serves, as seen in extracted text]
Keyword gap: [search intents or terms weakly covered on their page — note buyer language from reviews where possible]
Visual gap: [image or video proof types missing or weak on their gallery or A+]
Unmet buyer needs: [recurring themes from 3-star and mixed reviews, quoted or paraphrased]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🎯 YOUR STRATEGIC OPPORTUNITY POINTS (FOR YOUR BRAND OR ROADMAP — NOT EDITS TO THIS LISTING)

The ASIN above is the competitor under diagnosis. Below, translate gaps into **where you can win**; do not phrase outcomes as rewriting their bullets or their title.

Positioning and messaging whitespace:
  - [Claim, use case, or audience angle they under-own; why it is an opening for you]

Search and intent capture:
  - [Queries or intents implied by reviews or category that their listing weakly serves; how you could own a different slice of demand]

Trust, proof, and creative differentiation:
  - [Proof points, demos, or gallery angles they lack that you could credibly own]

Product, offer, or bundle opportunity:
  - [Unmet needs from reviews that map to a SKU, variant, bundle, warranty, or service on your side — stay factual to extracted complaints and wishes]

Competitive strengths to respect or neutralize:
  - [What this competitor does so well in copy, visuals, or social proof that you should assume as the bar before claiming superiority]

Amazon Product Search Skill

📖 Introduction

This skill utilizes BrowserAct's Amazon Product API template to extract structured product listings from Amazon search results. It provides detailed information including titles, ASINs, prices, ratings, and product specifications, enabling efficient market research and product monitoring without manual data collection.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The agent should configure the following parameters based on user requirements:

1. KeyWords

   • Type: string
   • Description: Search keywords used to find products on Amazon.
   • Required: Yes
   • Example: laptop, wireless earbuds

2. Brand

   • Type: string
   • Description: Filter products by brand name.
   • Default: Apple
   • Example: Dell, Samsung

3. Maximum_number_of_page_turns

   • Type: number
   • Description: Number of search result pages to paginate through.
   • Default: 1

4. language

   • Type: string
   • Description: UI language for the Amazon browsing session.
   • Default: en
   • Example: zh-CN, de

🚀 Usage

Agent should use the following independent script to achieve "one-line command result":

# Example Usage
python -u ./scripts/amazon_product_api.py "keywords" "brand" pages "language"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

Upon success, the script parses and prints the structured product data from the API response, which includes:

• product_title: Full title of the product.
• asin: Amazon Standard Identification Number.
• product_url: URL of the Amazon product page.
• brand: Brand name.
• price_current_amount: Current price.
• price_original_amount: Original price (if applicable).
• rating_average: Average star rating.
• rating_count: Total number of ratings.
• featured: Badges like "Best Seller" or "Amazon's Choice".
• color, material, style: Product attributes (if available).

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Market Research: Search for a specific product category to analyze top brands and pricing.
2. Competitor Monitoring: Track product listings and price changes for specific competitor brands.
3. Product Catalog Enrichment: Extract structured details like ASINs and specifications to build or update a product database.
4. Rating Analysis: Find high-rated products for specific keywords to identify market leaders.
5. Localized Research: Search Amazon in different languages to analyze international markets.
6. Price Tracking: Monitor current and original prices to identify discount trends.
7. Brand Performance: Evaluate the presence of a specific brand in search results across multiple pages.
8. Attribute Extraction: Gather technical specifications like material or color for a list of products.
9. Lead Generation: Identify popular products and their manufacturers for business outreach.
10. Automated Data Feed: Periodically pull Amazon search results into external BI tools or dashboards.

Amazon Product Search Automation Skill

📖 Introduction

This skill provides a one-stop product data collection service through BrowserAct's Amazon Product Search API template. It directly extracts structured product results from Amazon search lists. Simply input search keywords, brand filters, and quantity limits to get clean, usable product data.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure the following parameters based on user needs:

1. KeyWords (Search Keywords)

   • Type: string
   • Description: The keywords the user wants to search for on Amazon.
   • Example: phone, wireless earbuds, laptop stand

2. Brand (Brand Filter)

   • Type: string
   • Description: Filter products by brand name shown in the listing.
   • Example: Apple, Samsung, Sony

3. Maximum_date (Maximum Products)

   • Type: number
   • Description: The maximum number of products to extract across paginated search results.
   • Default: 50

4. language (UI Language)

   • Type: string
   • Description: UI language for the Amazon browsing session.
   • Options: en, de, fr, it, es, ja, zh-CN, zh-TW
   • Default: en

🚀 Usage

The Agent should execute the following independent script to achieve "one-line command result":

# Example Call
python -u ./scripts/amazon_product_search_api.py "Keywords" "Brand" Quantity "language"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script will parse and print results directly from the API response. Results include:

• product_title: Product name
• product_url: Detail page URL
• rating_score: Average star rating
• review_count: Total number of reviews
• monthly_sales: Estimated monthly sales (if available)
• current_price: Current selling price
• list_price: Original list price (if available)
• delivery_info: Delivery or fulfillment information
• shipping_location: Shipping origin or location
• is_best_seller: Whether marked as Best Seller
• is_available: Whether available for purchase

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Market Research: Search for "wireless earbuds" from "Sony" to analyze the current market.
2. Competitive Monitoring: Track "Samsung" phone prices and availability on Amazon.
3. Catalog Discovery: Gather product titles and URLs for a new product catalog in the "laptop stand" category.
4. Localized Analysis: Search Amazon in "ja" (Japanese) to understand products available in the Japan region.
5. Best Seller Tracking: Identify products marked as "Best Seller" for a specific brand.
6. Pricing Intelligence: Compare current_price and list_price to monitor discounts.
7. Sales Trend Estimation: Use monthly_sales data to estimate market demand for certain items.
8. Shipping Efficiency Study: Analyze delivery_info and shipping_location for various brands.
9. Large-scale Data Extraction: Collect up to 100 products for a comprehensive dataset.
10. Product Availability Check: Verify if specific brand products are currently is_available for purchase.

Amazon Reviews Automation Extraction Skill

📖 Introduction

This skill provides a one-stop Amazon review collection service through BrowserAct's Amazon Reviews API template. It can directly extract structured review results from Amazon product pages. By simply providing an ASIN, you can get clean, usable review data without building crawler scripts or requiring an Amazon account login.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure parameters based on user needs:

1. ASIN (Amazon Standard Identification Number)
   • Type: string
   • Description: The unique identifier for the product on Amazon.
   • Example: B07TS6R1SF, B08N5WRWJ6

🚀 Usage

The Agent should execute the following independent script to achieve "one-line command result":

# Example call
python -u ./scripts/amazon_reviews_api.py "ASIN_HERE"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script will parse and print results directly from the API response. Each review item includes:

• Commentator: Reviewer's name
• Commenter profile link: Link to the reviewer's profile
• Rating: Star rating
• reviewTitle: Headline of the review
• review Description: Full text of the review
• Published at: Date the review was published
• Country: Reviewer's country
• Variant: Product variant info (if available)
• Is Verified: Whether it's a verified purchase

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Competitor Analysis: Extract reviews for competitors' products to understand their strengths and weaknesses.
2. Product Feedback: Summarize feedback for your own products to identify areas for improvement.
3. Market Research: Collect data on customer preferences and common complaints in a specific category.
4. Sentiment Monitoring: Monitor recent reviews to detect shifts in customer sentiment.
5. QA Insights: Use customer reviews to identify potential quality issues or bugs.
6. Sentiment Analysis Prep: Gather review text and ratings for detailed emotion modeling.
7. Verified Purchase Analysis: Compare feedback from verified vs. unverified buyers.
8. Geographic Insights: Analyze product performance across different reviewer countries.
9. Variant Comparison: Understand which product variants (size/color) receive the best feedback.
10. Historical Trend Tracking: Retrieve and analyze review publication dates to track product lifecycle sentiment.

E-commerce — Product Listing

Category/search URL or keyword + filters → paginated product list (URL, name, price, image, rating per item)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract a structured list of products from any e-commerce category, search results, or keyword search page, with support for price/brand/rating filters and multi-page pagination.

Prerequisites

• Target browser is open and connected
• No login required for public listing pages

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". Use the bash tool for execution.

DOM: Extract product list from current page

Navigate to the listing/search page first, then extract:

eval "$(python scripts/extract-listing.py --max-results 20)"

Parameters:

• --max-results: max items to return per page, default 20

Output example:

{
  "count": 20,
  "items": [
    {
      "url": "https://www.amazon.com/dp/B09WNK39JN",
      "name": "Amazon Echo Pop",
      "price": 39.99,
      "currency": "USD",
      "image": "https://m.media-amazon.com/images/I/...jpg",
      "rating": 4.7,
      "review_count": 103789,
      "asin": "B09WNK39JN"
    }
  ]
}

DOM: Get next page URL

After extracting a page, get the URL to navigate to for the next page:

eval "$(python scripts/extract-listing-next-page.py)"

Output example:

{"next_url": "https://www.amazon.com/s?k=headphones&page=2", "has_next": true, "method": "amazon"}

When has_next is false, pagination is complete.

Composite: Keyword search with filters → product list

Step 1 — Build search URL with filters:

Construct the URL based on target site and desired filters using the patterns below, then navigate:

Amazon (amazon.com):

https://www.amazon.com/s?k={keyword_urlencoded}&s={sort}&rh={filter_params}

• Sort (s): price-asc-rank | price-desc-rank | review-rank | date-desc-rank (omit for relevance)
• Price filter: append p_36:{min_cents}-{max_cents} to rh (dollars × 100, e.g. $50–$200 → p_36:5000-20000)
• Rating filter: append avg_customer_review:four-and-above | three-and-above | two-and-above to rh
• In-stock: append p_n_availability:1248801011 to rh
• Multiple rh values: comma-separate (e.g. rh=p_36:5000-20000,avg_customer_review:four-and-above)

eBay (ebay.com):

https://www.ebay.com/sch/i.html?_nkw={keyword_urlencoded}&_udlo={min_price}&_udhi={max_price}&_sop={sort_num}

• Sort: 12=BestMatch | 15=PriceLow | 16=PriceHigh | 24=NewlyListed

Walmart (walmart.com):

https://www.walmart.com/search?q={keyword_urlencoded}&min_price={min}&max_price={max}&sort={sort}

• Sort: best_match | price_low | price_high | rating_high

Google Shopping (cross-site, no --site):

https://www.google.com/search?tbm=shop&q={keyword_urlencoded}&tbs=p_ord:{sort}

• Sort: rv=relevance | pd=price ascending | prd=price descending

Any site with --site (generic):

https://{site}/search?q={keyword_urlencoded}

Step 2 — Navigate and extract:

1. navigate {constructed_url} → wait stable
2. eval "$(python scripts/extract-listing.py --max-results {n})"

Step 3 — Paginate (repeat until done):

1. eval "$(python scripts/extract-listing-next-page.py)"
2. If has_next is true: navigate {next_url} → wait stable → re-run extract-listing.py
3. If has_next is false: stop

Pagination

URL Pagination: extract-listing-next-page.py detects rel=next link, platform-specific pagination controls, and URL page parameters. Returns next_url for navigation.

DOM Pagination: For sites with load-more buttons (some Shopify themes):

1. state to find "Load more" or "Show more" button
2. click <index> → wait stable → re-run extract-listing.py
3. Termination: button no longer present, or item count stops increasing

Success Criteria

result.count >= 1 AND items[0].url != null

Known Limitations

• Amazon: direct navigation may trigger bot detection on fresh sessions — navigate from https://www.amazon.com first
• eBay listing pages may require navigating from https://www.ebay.com first
• Google Shopping results have complex SPA structure and may have reduced accuracy; prefer direct site search when --site is specified
• Filter URL parameters are site-specific; unsupported filter parameters are silently ignored by some sites
• Shopify themes vary widely; if the generic DOM strategies miss items, check if the page has JSON-LD ItemList or Product array in page source

Execution Efficiency

• Batch orchestration: Loop through pages serially within a single session; add 1–2 second intervals between page navigations
• Test before batch execution: Test with 1 page before running multi-page extraction
• Error resumption: Record page number; on failure, resume from the last successful page

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/ecommerce-scraper-ecommerce-listing.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions; adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

E-commerce — Product Detail

Product URL / keyword / SKU → complete product data (name, price, brand, images, identifiers, availability, rating, variants)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract complete product information from any publicly accessible e-commerce product page using a universal multi-layer extraction strategy (JSON-LD → platform-specific DOM → OG meta → microdata).

Prerequisites

• Target browser is open and connected
• No login required for public product pages

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". Use the bash tool for execution.

DOM: Extract product data from current product page

Navigate to the product URL first, then extract:

eval "$(python scripts/extract-product.py)"

Output example:

{
  "url": "https://www.amazon.com/dp/B09WNK39JN",
  "name": "Amazon Echo Pop",
  "price": 39.99,
  "price_currency": "USD",
  "brand": "Amazon",
  "image": "https://m.media-amazon.com/images/I/61bTwy0ooPL.jpg",
  "images": ["https://...jpg", "https://...jpg"],
  "description": "Compact smart speaker with Alexa...",
  "category": ["Electronics", "Smart Speakers"],
  "sku": "B09WNK39JN",
  "gtin": null,
  "mpn": null,
  "availability": "InStock",
  "rating": 4.7,
  "review_count": 103789,
  "variants": [{"name": "Charcoal", "sku": "B09WNK39JN", "price": 39.99}],
  "seller": "Amazon",
  "identifiers": {"ASIN": "B09WNK39JN", "Best Sellers Rank": "#1 in Smart Speakers"},
  "_platform": "amazon",
  "_source": "json-ld"
}

Composite: Keyword or SKU → product detail

When input is a keyword, ASIN/SKU, or EAN/UPC rather than a direct product URL:

Step 1 — Navigate to search URL based on input type:

Step 2 — If landed on a search/listing page (multiple results):

1. wait stable
2. eval "$(python scripts/extract-listing.py --max-results 3)" — get top 3 results
3. Pick the most relevant product URL from items[0].url
4. navigate {product_url} → wait stable

Step 3 — Extract product data:

eval "$(python scripts/extract-product.py)"

Note: scripts/extract-listing.py is located in ../ecommerce-listing/scripts/extract-listing.py if used as a standalone Skill install; otherwise reference the listing Skill.

Success Criteria

result.name != null AND (result.price != null OR result.availability != null)

Known Limitations

• Amazon bot detection: direct navigation to a product URL may redirect to a CAPTCHA or bot-check page on fresh sessions. Navigate from https://www.amazon.com first to establish session cookies, then navigate to the product page
• eBay product pages may require navigating from https://www.ebay.com first; use solve-captcha if a challenge appears
• Some sites render product data entirely via client-side JavaScript; always use wait stable before extracting
• Price may be null for out-of-stock items or when login is required to view pricing
• Variant data completeness depends on whether the site includes full variant markup in JSON-LD

Execution Efficiency

• Batch orchestration: Write a bash script to loop through product URLs serially within a single session; add 1–2 second intervals between requests to avoid triggering anti-scraping restrictions
• Test before batch execution: Test with 1–2 URLs before running the full batch
• Error resumption: Save results item by item; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/ecommerce-scraper-ecommerce-product-detail.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

E-commerce — Product Reviews

Product URL → paginated customer reviews (reviewer, rating, date, title, body, verified, helpful votes)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract customer reviews from any publicly accessible e-commerce product or reviews page using a multi-strategy approach (JSON-LD Review → Amazon DOM → WooCommerce DOM → generic microdata → generic CSS patterns).

Prerequisites

• Target browser is open and connected
• No login required for public review pages

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". Use the bash tool for execution.

DOM: Extract reviews from current page

Navigate to the product/reviews page first, then extract:

eval "$(python scripts/extract-reviews.py --max-reviews 20)"

Parameters:

• --max-reviews: max reviews to return per page, default 20

Output example:

{
  "count": 20,
  "reviews": [
    {
      "reviewer": "John D.",
      "rating": 5.0,
      "date": "Reviewed in the United States on May 15, 2026",
      "title": "Great product, exactly as described",
      "body": "I've been using this for two weeks and it works perfectly...",
      "verified": true,
      "helpful_votes": 42
    }
  ]
}

Composite: Product URL → reviews with sort and pagination

Step 1 — Navigate to reviews page:

Step 2 — Extract reviews:

eval "$(python scripts/extract-reviews.py --max-reviews 20)"

Step 3 — Paginate (Amazon): Amazon review pages support URL pagination:

• Most recent sort: https://www.amazon.com/product-reviews/{ASIN}?sortBy=recent&pageNumber={page}
• Helpful sort: https://www.amazon.com/product-reviews/{ASIN}?sortBy=helpful&pageNumber={page}

For each page: navigate {reviews_url_with_page} → wait stable → re-run extract-reviews.py

Termination: when count returns 0, or no new reviews appear compared to prior page.

Pagination

URL Pagination (Amazon): Increment pageNumber parameter in the reviews URL. Start from 1.

DOM Pagination (WooCommerce/generic): Look for a "Next" pagination link on the reviews section. Use eval "$(python ../ecommerce-listing/scripts/extract-listing-next-page.py)" to detect it, then navigate.

Termination: has_next is false, or count is 0.

Success Criteria

result.count >= 1 AND reviews[0].body != null

Known Limitations

• Amazon: navigate from https://www.amazon.com first on fresh sessions to avoid bot detection
• JSON-LD reviews are often limited to a small subset (3–5 reviews) even when hundreds exist; use the Amazon-specific URL for full review extraction
• WooCommerce and Shopify review data depends on which review plugin is installed; body extraction may be null if a non-standard plugin is used
• Review dates may be locale-formatted strings rather than ISO dates depending on the site's configuration

Execution Efficiency

• Batch orchestration: Loop through review pages serially; add 1–2 second intervals between navigations
• Test before batch execution: Test with page 1 before running multi-page extraction
• Error resumption: Record page number; on failure, resume from last successful page

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/ecommerce-scraper-ecommerce-reviews.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions; adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

E-commerce — Seller Info

Seller/merchant profile URL → seller name, rating, review count, feedback, joined date, return policy

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract seller profile information from marketplace platform seller or storefront pages using JSON-LD structured data and platform-specific DOM patterns.

Prerequisites

• Target browser is open and connected
• No login required for public seller profile pages

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". Use the bash tool for execution.

DOM: Extract seller profile from current seller page

Navigate to the seller profile URL first, then extract:

eval "$(python scripts/extract-seller.py)"

Output example:

{
  "url": "https://www.amazon.com/shops/seller/A1234567890",
  "name": "TechGadgets Store",
  "description": "Premium electronics accessories since 2015",
  "rating": 4.8,
  "review_count": 12450,
  "positive_feedback_pct": "98% positive feedback",
  "joined": "Member since: January 2015",
  "return_policy": "30-day returns accepted",
  "image": null,
  "_platform": "amazon"
}

Composite: Amazon seller URL patterns

Amazon seller pages follow these URL patterns:

To find a seller from a product page:

1. Navigate to product page → wait stable
2. eval "document.querySelector('#sellerProfileTriggerId, #merchant-info a')?.href" to get the seller URL
3. navigate {seller_url} → wait stable
4. eval "$(python scripts/extract-seller.py)"

Composite: eBay seller URL patterns

To find seller from an eBay listing:

1. Navigate to eBay item page → wait stable
2. eval "document.querySelector('.x-sellercard-atf__data a[href*=\"/usr/\"]')?.href" to get seller URL
3. Navigate and extract

Success Criteria

result.name != null

Known Limitations

• Amazon seller pages may require navigating from https://www.amazon.com first on fresh sessions to avoid bot detection
• eBay seller pages may require navigating from https://www.ebay.com first
• Seller description and return policy availability depends on whether the seller has filled in their profile
• Rating scale differs by platform: Amazon uses 1–5 stars, eBay uses percentage of positive feedback; both are preserved in their native format

Execution Efficiency

• Batch orchestration: Loop through seller URLs serially; add 1–2 second intervals between navigations
• Test before batch execution: Test with 1–2 sellers before running the full batch
• Error resumption: Save results item by item; on failure, resume from the breakpoint

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/ecommerce-scraper-ecommerce-seller-info.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions; adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Goofish (闲鱼) — Item Detail

item URL (or item_id + category_id) → full listing data: title, price, seller info, description, images, tags, want-count

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Load a single Goofish item detail page and extract its complete listing data including seller information, item description, image gallery, and attribute tags.

Prerequisites

• Browser with an active Goofish session (login required — item detail pages require authenticated access)
• Target item URL format: https://www.goofish.com/item?id={item_id}&categoryId={category_id}

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Goofish has been confirmed in the current session → skip this step.

Otherwise: open https://www.goofish.com/ and observe the page:

• User avatar or account entry exists → logged in, continue
• Login/register prompt → not logged in; inform user that login is required; assist login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed on the page. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; use the bash tool for execution.

Network Capture: load item detail page

The item detail API (mtop.taobao.idle.pc.detail/1.0/) auto-fires when navigating to the item URL. Provide parameters via URL:

1. navigate https://www.goofish.com/item?id={item_id}&categoryId={category_id}
2. wait stable
3. Proceed to DOM extraction below

Error handling:

• If a CAPTCHA slider appears ("Please slide to verify"): the session is rate-limited. Wait 5–10 minutes, or use remote-assist to complete the slider manually, then re-navigate.
• If "网络不见了" error page appears: the item page API call failed. Retry once after 30 seconds; if it persists, the session may be temporarily blocked.
• If item shows "该宝贝已下架" or similar: item has been removed/sold — skip and move to next item.

Note: Navigating to item detail pages in rapid succession (e.g., less than 2 seconds apart) increases the probability of CAPTCHA. Add 2–5 second delays between items in batch mode.

DOM: extract item detail data

After navigating and waiting stable, extract all available fields:

eval "$(python scripts/extract-item-detail.py)"

Output example:

{
  "item_id": "1054899470781",
  "item_url": "https://www.goofish.com/item?id=1054899470781&categoryId=126862528",
  "title": "出一台iPhone 15 128G，电池健康度高，成色九新以上...",
  "price": "898.00",
  "original_price": "899.00",
  "seller_name": "小王数码严选",
  "seller_avatar": "https://img.alicdn.com/bao/uploaded/...",
  "seller_labels": ["成都", "刚刚擦亮", "来闲鱼5年", "卖出204件宝贝", "好评率100%"],
  "description": "出一台iPhone 15 128G，电池健康度高，成色九新以上...",
  "images": [
    "https://img.alicdn.com/bao/uploaded/i4/...",
    "https://img.alicdn.com/bao/uploaded/i2/..."
  ],
  "tags": ["品牌：Apple/苹果", "型号：iPhone 15", "存储容量：128GB", "运行内存：6GB", "成色：几乎全新"],
  "want_count": "8人想要"
}

Fields that may be null: original_price, seller_labels, description, tags, want_count (depending on listing completeness). Note: on Goofish detail pages, title and description contain the same text — there is no separate title element.

Pagination

N/A — single item page, no pagination.

Success Criteria

item_id non-null and title non-null and price non-null

Known Limitations

• Seller user ID (numeric) is not exposed in the DOM — only seller display name is available
• Item detail pages trigger CAPTCHA if accessed too rapidly; recommended minimum interval: 3 seconds between items
• Some items may require login even for viewing; the Skill will return an error if the page fails to load
• Rapid successive navigation may trigger "网络不见了" error — wait 30 seconds before retrying

Execution Efficiency

• Batch orchestration: Loop through item IDs serially with 3–5 second delays; do not parallelize within one browser. To increase throughput, distribute items across multiple browser sessions
• Test before batch execution: Test with 2–3 items first to confirm selectors are valid; only then run the full batch
• Error resumption: Save results item-by-item; on failure (CAPTCHA or error), record the failed item ID and resume after the CAPTCHA clears
• Get item IDs from search: Use goofish-search-list Skill to collect item IDs, then feed them into this Skill for detail enrichment

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/xianyu-scraper-goofish-item-detail.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file.

Goofish (闲鱼) — Search Results List

keyword + optional filters → list of 30 second-hand item cards per page (id, title, price, image, location, want-count)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract second-hand item listing cards from Goofish keyword search results, supporting sort options, price range filters, and publish-date filters, with page-by-page pagination.

Prerequisites

• Browser with an active Goofish session (logged-in account recommended for full results)
• Target page is already open or will be opened: https://www.goofish.com/search?q={keyword}

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for Goofish has been confirmed in the current session → skip this step.

Otherwise: open https://www.goofish.com/ and observe the page:

• User avatar or account entry exists → logged in, continue
• Login/register prompt → not logged in; inform user that login may be required for full results; assist login if needed

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Network Capture: trigger search and load results

Search requests use a dynamic sign token computed client-side — they cannot be reconstructed directly. Navigate to the search URL to trigger the API automatically.

1. navigate https://www.goofish.com/search?q={keyword}
2. wait stable
3. Proceed to DOM extraction below

Error handling: If the page shows a CAPTCHA slider ("Please slide to verify") instead of search results, the session has been rate-limited. Wait 5–10 minutes before retrying, or switch to a fresh browser session.

DOM: search result item cards (data extraction)

After navigating and waiting stable, extract all 30 item cards on the current page:

eval "$(python scripts/extract-search-items.py)"

Output example:

{
  "items": [
    {
      "item_id": "1054668718340",        // unique item ID
      "category_id": "126862528",        // category ID
      "item_url": "https://www.goofish.com/item?id=1054668718340&categoryId=126862528",
      "title": "美版iPhone 14 国行256G 纯原 原版原漆",  // full title text
      "image_url": "https://img.alicdn.com/bao/uploaded/...",  // thumbnail URL
      "price": "1810",                   // numeric string, CNY, no ¥ sign
      "service_tag": "Apple/苹果256GB无任何维修",  // condition/attribute tag or recency label, null if absent
      "price_desc": "2人想要",           // want-count or price-drop info, null if absent
      "location": "广东"                 // seller's location province/city
    }
  ],
  "count": 30
}

DOM: apply sort and filter options (operation)

Apply sort order, publish-date filter, or price range before extracting. Call before running extract-search-items.py. After calling, wait stable before extracting.

eval "$(python scripts/apply-search-filters.py --sort {sort} --publish-days {days} --price-min {min} --price-max {max})"

Parameters:

• --sort: Sort option — "" default (综合), "reduce" price-drop (新降价), "create" newest (新发布), "price-asc" price low-to-high, "price-desc" price high-to-low. Default: ""
• --publish-days: Filter by publish date — "" all, "1" within 1 day, "3" within 3 days, "7" within 7 days, "14" within 14 days. Default: ""
• --price-min: Minimum price (CNY integer string, e.g., "500"). Requires --price-max. Default: ""
• --price-max: Maximum price (CNY integer string, e.g., "3000"). Requires --price-min. Default: ""

Output example:

{
  "ok": true,
  "applied": {
    "sort": "reduce:desc",
    "searchFilter": "publishDays:7;priceRange:500,3000;"
  }
}

DOM: navigate to a specific page (operation)

eval "$(python scripts/goto-page.py {page_number})"

Parameters:

• page_number: Target page number (integer, 1-based)

Output example:

{ "ok": true, "clicked_page": 2 }

After clicking, wait stable then re-run extract-search-items.py to get the new page's items.

Enum Parameters

[AI] sort options: "" (综合/default), "reduce" (新降价), "create" (新发布/最新), "price-asc" (价格从低到高), "price-desc" (价格从高到低)

[AI] publish-days filter: "" (all), "1", "3", "7", "14"

Pagination

DOM Pagination: Click the target page number button using goto-page.py {page}, then wait stable, then re-run extract-search-items.py. Page numbers appear in the pagination bar at the bottom of the search results.

Termination: When goto-page.py returns error: Page N not found — no more pages available, or the target page exceeds the pagination range displayed (typically up to 25 pages / 750 items).

Success Criteria

result count >= 1 and item_id non-null rate = 100% and price non-null rate >= 80%

Known Limitations

• 30 items per page (fixed by the site)
• Maximum ~750 items accessible via pagination (25 pages × 30)
• Seller username and user ID are not available in search cards — only seller location
• Session rate limiting: accessing item detail pages rapidly after heavy search usage may trigger a CAPTCHA slider; mitigate by adding 1–2 second delays between page navigations
• The sign token in search API requests is computed client-side; direct API replay without browser context is not supported — always trigger via page navigation

Execution Efficiency

• Batch orchestration: Write a bash script to loop through keywords serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping). Add 1–2 second delays between page navigations. To increase throughput, open multiple stealth browser sessions and distribute keywords across them
• Test before batch execution: After writing a batch script, first test with 1–2 keywords/pages to verify the script runs correctly; only then run the full batch
• Reduce redundant pre-operations: Navigate once per keyword, apply all filters at once before extracting, rather than navigating multiple times
• Error resumption: Save results keyword-by-keyword and page-by-page during batch processing; on failure, resume from the last saved position

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/xianyu-scraper-goofish-search-list.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Business Contact & Social Links Extractor Skill

📖 Brief

This skill integrates two BrowserAct templates to cover two common scenarios. When the user provides a company name, it searches Google to extract the company's official website and social media channels. When the user provides a website URL, it scrapes the site to extract social media profile links (LinkedIn, Facebook, X/Twitter, Instagram, YouTube, TikTok). The script auto-detects the input type and routes to the correct template.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key yet, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The agent should determine whether the input is a company name or a website URL and pass it to the script. The script automatically routes the request to the correct template.

1. Target (Company Name or Website URL)
   • Required: Yes
   • Type: string
   • Description: The name of the company to search for, or the direct website URL to scrape social links from.
   • Example: OpenAI or https://www.rotorooter.com/

🚀 Invocation Method

Agent should execute the following command to invoke the skill:

# Search by company name
python -u ./scripts/business_contact_social_links.py "OpenAI"

# Scrape by website URL
python -u ./scripts/business_contact_social_links.py "https://www.rotorooter.com/"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take several minutes. The script outputs timestamped status logs continuously (e.g., [14:30:05] Task Status: running). Agent guidelines:

• Monitor the terminal output while waiting.
• As long as new status logs appear, the task is running normally; do not misjudge it as frozen.
• Only consider triggering retry if the status remains unchanged for a long time or output stops without a final result.

📊 Data Output

Upon successful execution, the script prints structured JSON data extracted from the API response.

If a Company Name was provided, the output includes:

• Company Name: The official name of the company.
• Current Page URL: The official website URL.
• Social Media Information: A consolidated list of discovered social media profiles.

If a Website URL was provided, the output includes an array of objects:

• platform: Platform name (e.g., LinkedIn, YouTube).
• url: Canonical URL of the external profile.

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output contains "concurrent" or "too many running tasks", it means the concurrent task limit has been reached. Do not retry; guide the user to upgrade their plan. Agent must inform the user:

     "The current task cannot be executed because your BrowserAct account has reached the concurrent task limit. Please visit the BrowserAct Plan Upgrade Page to upgrade your plan."

   • If the output does not contain the above error keywords but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error to the user.

🌟 Typical Use Cases

1. Sales Lead Enrichment: Find the official website and social channels for a list of company names.
2. Influencer Outreach: Extract all social media profiles from a creator's personal website.
3. Competitor Analysis: Discover which social media platforms a rival company prioritizes.
4. CRM Data Cleaning: Verify and populate missing social links and official URLs for existing leads.
5. Multi-Channel Marketing: Build targeted lists of businesses with active social media presence.
6. Social Listening Setup: Quickly gather official handles to monitor brand mentions.
7. B2B Prospecting: Combine with Yellow Pages data to build complete business intelligence profiles.
8. Partner Vetting: Evaluate the digital footprint of a potential business partner using just their name.
9. Directory Building: Automate the collection of website and social links for a local business directory.
10. Brand Tracking: Track all official contact channels of specific brands across the web.

GitHub Project & Contributor Finder API Skill

📖 Brief

This skill utilizes BrowserAct's GitHub Project & Contributor Finder API to extract project details and contributor contact information from GitHub. Simply provide keywords, minimum stars, and an update date filter — BrowserAct traverses the search results, extracts repository metrics, and fetches detailed contributor profiles, returning it all directly via API without building crawler scripts or dealing with rate limits.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key yet, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The agent should flexibly configure the following parameters based on user requirements:

1. KeyWords

   • Type: string
   • Description: Keywords for searching repositories.
   • Example: browser automation, react framework, machine learning
   • Default: browser automation

2. stars

   • Type: number
   • Description: Minimum number of stars the repository should have.
   • Example: 100, 1000
   • Default: 100

3. updated

   • Type: string
   • Description: Filter repositories by the date they were last updated (format: YYYY-MM-DD).
   • Example: 2026-01-01, 2025-06-01
   • Default: 2026-01-01

4. Page_Turns

   • Type: number
   • Description: Number of search result pages to paginate through. For example, if there are 39 pages and you want the first 2, input 2.
   • Example: 1, 2
   • Default: 1

5. date_limit_per_page

   • Type: number
   • Description: Number of data items to extract per page in the search results list.
   • Example: 5, 10
   • Default: 5

🚀 Invocation Method

Agent should execute the following command to invoke the skill:

# Example invocation (all parameters)
python -u ./scripts/github_project_contributor_finder_api.py "browser automation" 100 "2026-01-01" 1 5

# Minimal invocation (only keywords, others use defaults)
python -u ./scripts/github_project_contributor_finder_api.py "react framework"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take several minutes. The script outputs timestamped status logs continuously (e.g., [14:30:05] Task Status: running). Agent guidelines:

• Monitor the terminal output while waiting.
• As long as new status logs appear, the task is running normally; do not misjudge it as frozen.
• Only consider triggering retry if the status remains unchanged for a long time or output stops without a final result.

📊 Data Output

Upon successful execution, the script parses and prints the structured results from the API response.

Project Fields:

• repository_name: The name of the GitHub repository.
• repository_url: The URL link to the repository.
• repository_owner_name: The owner/creator of the repository.
• repository_description: A brief description of the repository.
• star_count: The number of stars the repository has received.

Contributor Fields:

• user_name: The GitHub username of the contributor.
• profile_url: The URL link to the contributor's profile.
• bio: The bio or short description of the contributor.
• repositories_summary: A summary of other repositories owned by the contributor.
• personal_website: The contributor's personal website link.
• twitter: The contributor's Twitter handle.

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output contains "concurrent" or "too many running tasks", it means the concurrent task limit has been reached. Do not retry; guide the user to upgrade their plan. Agent must inform the user:

     "The current task cannot be executed because your BrowserAct account has reached the concurrent task limit. Please visit the BrowserAct Plan Upgrade Page to upgrade your plan."

   • If the output does not contain the above error keywords but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error to the user.

🌟 Typical Use Cases

1. Tech Recruiting: Gather contributor profiles and social links from popular repositories to build candidate pipelines.
2. Open-Source Discovery: Search for trending repositories by keywords and star count to find valuable projects.
3. Developer Outreach: Collect personal websites and Twitter handles of active contributors for developer tool marketing.
4. Community Building: Identify and connect with active open-source contributors in specific domains.
5. Competitor Analysis: Monitor which developers contribute to competing projects.
6. Partnership Scouting: Find repository owners for potential collaboration or sponsorship.
7. Market Research: Analyze repository metrics and descriptions to understand technology trends.
8. Lead Generation: Generate targeted leads for developer tools by finding projects with relevant tech stacks.
9. Academic Research: Discover recently updated repositories in specific research areas.
10. Talent Mapping: Build a database of skilled developers based on their GitHub contributions and profiles.

Google Maps Automation Scraper Skill

📖 Introduction

This skill leverages BrowserAct's Google Maps API template to provide a one-stop business data collection service. It extracts structured details directly from Google Maps, including business names, categories, contact info, ratings, and more. Simply provide the search keywords and location bias to get clean, actionable data.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

Configure the following parameters based on user requirements:

1. keywords (Search Keywords)

   • Type: string
   • Description: The query you would search for on Google Maps.
   • Example: coffee shop, dental clinic, Turkish-style restaurant

2. language (UI Language)

   • Type: string
   • Description: Defines the UI language and returned text language (e.g., en, zh-CN).
   • Default: en

3. country (Country Bias)

   • Type: string
   • Description: Specifies the country or region bias (e.g., us, gb, ca).
   • Default: us

🚀 Usage

Execute the following script to get results in one command:

# Example call
python -u ./scripts/google_maps_api.py "keywords" "language" "country"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

Upon success, the script parses and prints the following fields from the API:

• Title Name: Official business name
• Category_primary: Main business category
• Address: Full street address
• Phone number: Contact phone number
• Website link: Official URL
• Rating: Average star rating
• reviews_count: Total number of reviews
• business_status: Operational status (e.g., operational)

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Lead Generation: Find "SaaS companies" in "us" for sales outreach.
2. Competitor Research: Extract data on "coffee shops" in a specific neighborhood.
3. Market Analysis: Identify the density of "dental clinics" in a region.
4. Contact Info Retrieval: Get phone numbers and websites for "real estate agencies".
5. Local Service Discovery: Find "Turkish-style restaurants" with high ratings.
6. Business Status Monitoring: Check if specific stores are "operational".
7. Directory Building: Gather addresses and categories for a local business directory.
8. Rating Benchmarking: Compare ratings of various "luxury hotels".
9. Global Scouting: Research "tech startups" in different countries like "gb" or "au".
10. Automated Data Sync: Periodically pull local business data into a CRM.

Google Maps — Contact Extractor

keyword + location → business list with name/address/phone/website + email/social media/contacts from each business's website

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Search Google Maps by keyword and location, collect place details from each result, visit each business's website to extract emails and social media profiles, and return a merged dataset replicating Google Maps Email Extractor functionality.

Prerequisites

• For search: navigate to https://www.google.com/maps/search/{keyword}/@{lat},{lng},{zoom}z — the search results sidebar (feed) must be visible
• For place detail: navigate to the individual place URL https://www.google.com/maps/place/?q=place_id:{place_id} — the place sidebar with name, address, phone must be visible
• For website contact extraction: navigate to the business's website homepage
• No login required for Google Maps or most business websites

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". It is recommended to use the bash tool for execution. Working directory: all python scripts/ commands must be run from the Skill's root directory (the directory containing SKILL.md and scripts/). Use cd {skill-directory} before running batch scripts, or use absolute paths: python {absolute-path-to-scripts}/xxx.py.

DOM: Search results list — extract business cards from Google Maps search feed

Prerequisite: navigate to and wait for Google Maps search results to load, then extract all visible business cards.

navigate https://www.google.com/maps/search/{keyword}/@{lat},{lng},{zoom}z
wait stable
eval "$(python scripts/search-places.py '{keyword}' --max {max_count})"

Parameters:

• {keyword}: search term, e.g. coffee shops
• --max: maximum number of places to extract, default 20
• {lat},{lng}: center coordinates, e.g. 40.7580,-73.9855
• {zoom}: zoom level, e.g. 14

Output example:

{
  "keyword": "coffee shops",
  "count": 20,
  "places": [
    {
      "name": "Blue Dove Coffee",
      "rating": 4.8,
      "review_count": "292",
      "category": "Coffee shop",
      "price_range": null,
      "phone": null,
      "open_status": "Closed · Opens 7 AM",
      "place_id": "0xa5922b78cc420fb1:0x535506dc9cdb2cec",
      "lat": 40.7368708,
      "lng": -73.9909297,
      "maps_url": "https://www.google.com/maps/place/Blue+Dove+Coffee/data=..."
    }
  ]
}

Pagination: Google Maps loads ~20 results per view. Scroll down in the results panel to trigger loading of additional results, then re-run the extraction script. Repeat until desired count is reached.

scroll down --amount 3000
wait stable
eval "$(python scripts/search-places.py '{keyword}' --max {total_count})"

DOM: Place detail — extract full business info from a Google Maps place page

Prerequisite: navigate to the place page and wait for the sidebar to load with name, address, and phone visible.

navigate https://www.google.com/maps/place/?q=place_id:{place_id}
wait stable
wait --selector "h1" --state visible --timeout 15000
eval "$(python scripts/place-detail.py)"

Alternatively, navigate directly using the maps_url returned from the search results component.

Output example:

{
  "place_id": "0xa5922b78cc420fb1:0x535506dc9cdb2cec",
  "name": "Blue Dove Coffee",
  "rating": 4.8,
  "review_count": 292,
  "category": "Coffee shop",
  "address": "33 Union Square W, New York, NY 10003",
  "located_in": null,
  "phone": "(646) 939-0937",
  "website": "https://www.bluedovecoffee.com/",
  "menu_url": "https://order.dripos.com/Blue-Dove-Coffee",
  "price_range": "$1–10",
  "coordinates": { "lat": 40.7368708, "lng": -73.9909297 },
  "hours": [
    "Monday7 AM–5 PM",
    "Tuesday7 AM–5 PM",
    "Wednesday7 AM–5 PM",
    "Thursday7 AM–5 PM",
    "Friday7 AM–5 PM",
    "Saturday7 AM–5 PM",
    "Sunday7 AM–4 PM"
  ],
  "service_options": ["Serves dine-in", "Offers takeout", "Offers delivery"],
  "amenities": ["Has wheelchair accessible entrance", "Accepts credit cards", "Accepts NFC mobile payments"],
  "maps_url": "https://www.google.com/maps/place/..."
}

DOM: Website contacts — extract emails, phones, social media from a business website

Prerequisite: navigate to the business website homepage.

navigate {website_url}
wait stable
eval "$(python scripts/extract-contacts.py --depth shallow)"

For deeper extraction (also scans contact/about subpages), use --depth deep. After running with --depth deep, check contact_subpages in the output, then navigate to each and re-run:

navigate {subpage_url}
wait stable
eval "$(python scripts/extract-contacts.py --depth shallow)"

Parameters:

• --depth: shallow (homepage only, default) or deep (also discovers and lists contact/about subpage URLs)

Output example:

{
  "source_url": "https://www.bluedovecoffee.com/",
  "emails": ["support@bluedovecoffee.com", "careers@bluedovecoffee.com", "orders@bluedovecoffee.com"],
  "phone_numbers": [],
  "social_media": {
    "facebook": ["https://www.facebook.com/people/Blue-Dove-Coffee/100094867009022"],
    "instagram": ["https://www.instagram.com/bluedovecoffee"],
    "tiktok": ["https://www.tiktok.com/@bluedovecoffee"]
  },
  "contact_subpages": []
}

Composite: Full pipeline — search + place details + website contacts

Complete flow replicating Google Maps Email Extractor:

1. Navigate to Google Maps search:

   navigate https://www.google.com/maps/search/{keyword}/@{lat},{lng},{zoom}z
   wait stable
   eval "$(python scripts/search-places.py '{keyword}' --max {max_count})"
   

   Save list of places (especially name, maps_url, place_id).

2. For each place: navigate to place detail page and extract full info:

   navigate {maps_url}
   wait stable
   wait --selector "h1" --state visible --timeout 15000
   eval "$(python scripts/place-detail.py)"
   

   Collect website, phone, address, hours, rating, etc.

3. For each place with a website value: extract contacts from the website:

   navigate {website}
   wait stable
   eval "$(python scripts/extract-contacts.py --depth deep)"
   

   For each URL in contact_subpages, navigate and extract again to find additional emails.

4. Merge results: join by place_id or name. Final record per business:

   {
     "name": "Blue Dove Coffee",
     "address": "33 Union Square W, New York, NY 10003",
     "phone": "(646) 939-0937",
     "website": "https://www.bluedovecoffee.com/",
     "rating": 4.8,
     "review_count": 292,
     "category": "Coffee shop",
     "coordinates": { "lat": 40.7368708, "lng": -73.9909297 },
     "hours": ["Monday7 AM–5 PM", "..."],
     "service_options": ["Serves dine-in", "Offers takeout"],
     "emails": ["support@bluedovecoffee.com"],
     "social_media": {
       "instagram": ["https://www.instagram.com/bluedovecoffee"],
       "tiktok": ["https://www.tiktok.com/@bluedovecoffee"]
     }
   }
   

Pagination

DOM Pagination: Google Maps search loads ~20 results initially. Scroll down in the sidebar:

scroll down --amount 3000
wait stable
eval "$(python scripts/search-places.py '{keyword}' --max {total_count})"

Repeat until the desired number of results is reached. Termination: Google Maps typically shows up to ~120 results per search; the sidebar will show "You've reached the end of the results" or stop loading new items.

Success Criteria

• places.count >= 1 from search results extraction
• Place detail returns name != null AND (address != null OR phone != null)
• Website contact extraction returns emails.length + phone_numbers.length + Object.keys(social_media).length >= 0 (no contacts = valid result for businesses without public contact info)

Known Limitations

• Google Maps search results are limited to approximately 120 businesses per search query; to get more, use more specific sub-area searches
• Phone numbers on the Google Maps search list card are not always shown; place-detail.py on the individual place page is more reliable
• Website contact extraction depends on the business's website structure; sites using image-based emails or obfuscated text will return no emails
• Some businesses do not have a website listed on Google Maps; those cannot be enriched with contact data
• Google Maps may throttle requests if too many place pages are navigated in rapid succession; add 1–2 second delays between place navigations in batch scripts

Execution Efficiency

• Batch orchestration: Write a bash script that loops through the pipeline steps in a single session. Add a 1–2 second sleep between each place detail navigation to avoid triggering anti-scraping. Example:

  for place_url in "${place_urls[@]}"; do
    browser-act --session gmaps-s1 navigate "$place_url"
    browser-act --session gmaps-s1 wait stable
    browser-act --session gmaps-s1 eval "$(python scripts/place-detail.py)"
    sleep 1.5
  done
  

• Test before batch execution: Test with 2–3 places before running the full batch
• Error resumption: Save each place's result to a JSON file immediately after extraction; on failure, skip already-saved places by checking if the file exists
• Multiple sessions for throughput: Open 2–3 parallel stealth browser sessions and distribute places across them; each session has an independent fingerprint

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/google-maps-contact-extract.memory.md

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file.

Google Maps Reviews Automation Skill

📖 Introduction

This skill provides a one-stop review collection service using BrowserAct's Google Maps Reviews API template. It can extract structured review data directly from Google Maps search results. Simply provide the search keywords, language, and country to get clean, usable review data.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The Agent should flexibly configure the following parameters when calling the script:

1. KeyWords (Search Keywords)

   • Type: string
   • Description: The query used to find places on Google Maps (e.g., business names, categories).
   • Example: coffee shop, dental clinic, Tesla showroom

2. language (Language)

   • Type: string
   • Description: Sets the UI language and the language of the returned text.
   • Supported values: en, zh-CN, es, fr, etc.
   • Default: en

3. country (Country)

   • Type: string
   • Description: Country or region bias for search results.
   • Supported values: us, gb, ca, au, jp, etc.
   • Default: us

🚀 Usage

Agent should use the following independent script to achieve "one-line command result":

# Example call
python -u ./scripts/google_maps_reviews_api.py "Keywords" "Language" "Country"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script parses and prints results from the API response:

• author_name: Display name of the reviewer
• author_profile_url: Profile URL of the reviewer
• rating: Star rating
• text: Review text content
• comment_date: Human-readable date
• likes_count: Number of likes
• author_image_url: Reviewer's avatar URL

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Local Business Analysis: Find reviews for cafes or clinics in a specific area.
2. Reputation Monitoring: Track feedback for a specific brand location.
3. Competitive Benchmarking: Analyze reviews of competitor stores.
4. Sentiment Analysis: Gather review text for emotion and topic modeling.
5. Market Research: Evaluate service quality across different regions.
6. Lead Qualification: Use review data to identify high-quality service providers.
7. Customer Insight: Understand recurring complaints or praises.
8. Venue Research: Collect testimonials for parks, museums, or attractions.
9. Retail Monitoring: Gather feedback for newly opened stores.
10. Service Quality Audit: Analyze ratings and comments for a specific service chain.

Google Maps Search Automation Skill

📖 Introduction

This skill utilizes the BrowserAct Google Maps Search API template to provide a one-stop business data collection service. It extracts structured business results directly from Google Maps search lists. Simply provide search keywords, language, and country filters to get clean, usable business data.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

When calling the script, the Agent should flexibly configure the following parameters based on user needs:

1. KeyWords

   • Type: string
   • Description: The content you want to search on Google Maps. Can be business names, categories, or specific keywords.
   • Example: coffee shop, dental clinic, coworking space

2. language

   • Type: string
   • Description: Sets the UI language and the language of returned text fields.
   • Common Values: en, de, fr, it, es, ja, zh-CN, zh-TW
   • Default: en

3. country

   • Type: string
   • Description: Sets the country or region bias for search results.
   • Example: us, gb, ca, au, de, fr, jp
   • Default: us

4. max_dates

   • Type: number
   • Description: The maximum number of places to extract from the search results list.
   • Default: 100

🚀 Usage

The Agent should execute the following independent script to achieve "one-line command result":

# Example call
python -u ./scripts/google_maps_search_api.py "search keywords" "language" "country" count

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script parses and prints results directly from the API response. Fields include:

• name: Business name
• full address: Full address
• rating: Star rating
• review count: Number of reviews
• price range: Price level
• cuisine type: Business category
• amenity tags: Features like Wi-Fi
• review snippet: Short review text
• service options: Service indicators (e.g., "Order online")

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Local Business Discovery: Find all "Italian restaurants" in Manhattan.
2. Sales Lead Generation: Extract a list of "real estate agencies" in London for prospecting.
3. Competitor Mapping: Locate all "Starbucks" branches in a specific city to map competition.
4. Market Research: Analyze "boutique hotels" in Paris, including their ratings and price ranges.
5. Contact Collection: Gather addresses and names of "dental clinics" in Sydney.
6. Service Search: Find "24-hour pharmacies" or "emergency plumbers" in a certain area.
7. Neighborhood Monitoring: Track new "gyms" or "yoga studios" opening in a community.
8. Structured Data Export: Export structured data of "car dealerships" for CRM integration.
9. Sentiment Analysis Prep: Get review snippets and ratings for "popular tourist attractions".
10. Global Search Localization: Use different language and country settings to research "tech hubs" globally.

Google — Social Media Finder

Name or brand → all social media profiles found on Google (platform, URL, username, bio, followers)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Given a person's name, brand name, or username, search Google and return all matching social media profile results from known platforms.

Prerequisites

• No login required — Google search is publicly accessible

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

DOM: social media profile results (data extraction type)

Navigate to the Google search page for the target name, then extract all social media profile results.

Step 1 — Navigate:

navigate https://www.google.com/search?q={name}+social+media

Replace {name} with the person's or brand's name, using + in place of spaces (e.g., Taylor+Swift, Elon+Musk, Nike).

Step 2 — Wait for page:

wait stable

Step 3 — Extract:

eval "$(python scripts/extract-social-profiles.py)"

Output example:

{
  "error": false,
  "count": 5,
  "results": [
    {
      "platform": "Instagram",      // social media platform name
      "username": "taylorswift",    // handle or page name shown alongside platform
      "url": "https://www.instagram.com/taylorswift/",  // direct profile URL
      "title": "Taylor Swift (@taylorswift) • Instagram photos and videos",  // page title
      "snippet": "274M followers · 0 following · 706 posts ...",  // bio/description snippet from search result
      "followers": "超过 2.7亿位关注者"  // follower count as displayed (language depends on browser locale)
    }
  ]
}

On error: {"error": true, "message": "..."} — check that the browser navigated to a Google search page and .tF2Cxc result containers are present.

Pagination

URL Pagination: URL pattern https://www.google.com/search?q={name}+social+media&start={offset}, where offset = (page - 1) * 10 (page 1 → start=0 or omit, page 2 → start=10, page 3 → start=20). Next page link: a#pnnext. Termination: a#pnnext is absent (last page reached) or no social media results returned.

Success Criteria

result count >= 1 and platform and url fields are non-null for every item

Known Limitations

• Results depend on Google's index — newly created or low-traffic profiles may not appear
• Follower count text is localized to the browser's display language (e.g., Chinese characters for a Chinese-locale stealth browser)
• Google may show sub-pages of the same profile as separate results (e.g., both /elonmusk and /elonmusk/with_replies from X); deduplicate by base URL if needed
• Google SERP layout changes occasionally; if .tF2Cxc stops matching, inspect page HTML for updated container class names

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/social-media-finder-google-social-media-finder.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Indeed — Job Search

keyword + location + country → structured job listing data (title, company, salary, description, benefits, apply link)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract structured job listing data from Indeed search results including full job descriptions, salary information, company details, and application links.

Prerequisites

• Target page is already open in the browser: https://www.indeed.com/jobs?q={keyword}&l={location}
• For pagination beyond page 1: user must be logged into Indeed (login/sign-in button is NOT visible, user avatar or account menu IS visible)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification (for pagination beyond page 1)

If login status for Indeed has been confirmed in the current session → skip this step.

Otherwise: open Indeed and observe the page login status:

• Sign out entry, user avatar, or account menu exists → logged in, continue execution
• Sign in/Register entry exists with no sign out entry → not logged in, inform the user that login is needed for pagination beyond page 1, assist the user in completing the login flow

User refuses or cannot log in → can still extract page 1 results (up to 24 jobs per search), but cannot paginate.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: Extract job list from search results page

eval "$(python scripts/search-jobs.py --max-items {max_items})"

Parameters:

• --max-items: Maximum number of items to return from current page, 0 for all (default: 0)

Prerequisites: Browser must be on an Indeed search results page (indeed.com/jobs?q=...). Navigate first:

1. navigate https://www.indeed.com/jobs?q={keyword}&l={location} (add &start={offset} for pagination)
2. wait stable
3. If page title contains "Security Check" → solve-captcha → wait stable

Output example:

{
  "pageNumber": 1,
  "totalResults": 24,
  "results": [
    {
      "id": "8c06afbaf73fc880",
      "positionName": "Software Engineer",
      "company": "Google",
      "location": "San Francisco, CA",
      "postedAt": "3 days ago",
      "salary": "$120,000 - $180,000 a year",
      "salaryMin": 120000,
      "salaryMax": 180000,
      "salaryCurrency": "USD",
      "salaryType": "YEARLY",
      "jobType": ["Full-time"],
      "rating": 4.3,
      "reviewsCount": 15000,
      "companyLogo": "https://d2q79iu7y748jz.cloudfront.net/s/_squarelogo/256x256/...",
      "snippet": "<ul>...<b>software</b>...</ul>",
      "sponsored": false,
      "expired": false,
      "newJob": true,
      "url": "https://www.indeed.com/viewjob?jk=8c06afbaf73fc880",
      "externalApplyLink": null
    }
  ]
}

API: Fetch full job detail by job key

eval "$(python scripts/fetch-job-detail.py '{jobkey}')"

Parameters:

• jobkey: Indeed job key (the id field from search results)

Prerequisites: Browser must be on the same Indeed search results page where the job was found (the fetch uses same-origin cookies). Do NOT navigate away between search and detail fetch.

Output example:

{
  "id": "a39828c8395d5dfe",
  "positionName": "Sr. Application Developer",
  "company": "University of California San Francisco",
  "location": "San Francisco, CA 94158",
  "rating": 4,
  "reviewsCount": 709,
  "companyLogo": "https://d2q79iu7y748jz.cloudfront.net/s/_squarelogo/256x256/...",
  "salary": "$101,300 - $190,000 a year",
  "salaryMin": 101300,
  "salaryMax": 190000,
  "salaryCurrency": "USD",
  "salaryType": "YEARLY",
  "jobType": "Full-time",
  "description": "<p><b>Job Description:</b></p><p>...(full HTML)...</p>",
  "benefits": ["Health insurance", "401(k)", "Dental insurance"],
  "postedAt": "13 days ago",
  "isExpired": false, "url": "https://www.indeed.com/viewjob?jk=a39828c8395d5dfe"
}

Composite: Full job search with descriptions

Complete workflow to extract job listings with full descriptions:

1. navigate https://www.indeed.com/jobs?q={keyword}&l={location} → wait stable
2. If title contains "Security Check" → solve-captcha → wait stable
3. eval "$(python scripts/search-jobs.py)" → get job list with IDs
4. For each job id from step 3:
   • eval "$(python scripts/fetch-job-detail.py '{id}')" → get full description
5. Merge: search results provide base data, detail fetch adds description, benefits, enriched salary/rating

For pagination (requires login):

• Repeat steps 1-4 with &start=10, &start=20, &start=30, etc.
• Termination: when search-jobs.py returns fewer results than expected or error: true

Output: Combined array where each item has all fields from both search-jobs and fetch-job-detail.

Enum Parameters

[API] country — supported country codes for Indeed search (append to URL as domain variant https://{country-domain}/jobs?q=...):

For other countries, use format: {country-code-lowercase}.indeed.com

Pagination

URL Pagination: Parameter start, type: page-number (offset-based). Start value: 0 (page 1, implicit). Next page: increment by 10 (start=10, start=20, start=30...). Termination: search-jobs.py returns error: true with message about no results, or returns fewer than expected items. Note: each page actually returns ~15-24 items despite the offset increment of 10.

Login requirement: Page 2+ (start=10 and above) redirects to secure.indeed.com/auth if not logged in.

Success Criteria

• result count >= 1 from search-jobs.py
• Core fields non-null: id, positionName, company, location present for all results
• description field from fetch-job-detail.py is non-null HTML string with length > 100
• Data matches page content (job titles correspond to visible listings)

Known Limitations

• Pagination beyond page 1 requires Indeed login (unauthenticated access limited to first page, ~24 results)
• Indeed may trigger captcha/security check on first visit; solve-captcha resolves this
• Job detail fetch (fetch-job-detail.py) must execute from within the search results page context (same-origin requirement)
• Indeed's anti-bot detection may block access after excessive requests; use appropriate intervals (2-5 seconds between detail fetches)
• Some fields (salary, jobType, benefits) may be null when not provided by the employer

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Add 2-3 second intervals between detail fetches. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: Extract all job IDs from the search page first (one eval), then batch-fetch details — avoid re-navigating to the search page between fetches
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: browser-act-skill-forge-memories/indeed-jobs-scraper-indeed-job-search.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Industry Key Contact Radar

📖 Brief

This skill provides a one-stop contact discovery service using BrowserAct's Industry Key Contact Radar API template. It extracts structured contact details directly from search results across various platforms, including profile URLs, names, introductions, and company associations. Simply input an industry, result limit, target site, and job title to receive clean, actionable contact data.

✨ Features

1. No hallucinations, ensuring stable and accurate data extraction: Pre-set workflows avoid AI generative hallucinations.
2. No CAPTCHA issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP restrictions or geo-blocking: No need to handle regional IP restrictions or geofencing.
4. Faster execution: Tasks execute faster compared to purely AI-driven browser automation solutions.
5. Extremely high cost-efficiency: Significantly reduces data acquisition costs compared to AI solutions that consume massive amounts of tokens.

🔑 API Key Guide

Before running, you must check the BROWSERACT_API_KEY environment variable. If it is not set, do not take other actions first; you should ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key yet, please go to the BrowserAct Console to get your Key."

🛠️ Input Parameters

Agent should flexibly configure the following parameters based on user needs:

1. industry (Industry)

   • Type: string
   • Description: The industry or field to search for contacts.
   • Example: Browser automation, E-commerce, Healthcare
   • Default: Browser automation
   • Required: Yes

2. Datelimit (Max Items)

   • Type: number
   • Description: Maximum number of records to extract. The default value is recommended for the first run.
   • Example: 10
   • Default: 10

3. site (Target Site)

   • Type: string
   • Description: The social platform or website to search on.
   • Example: facebook.com, linkedin.com, github.com
   • Default: facebook.com
   • Required: Yes

4. Job_Title (Job Title)

   • Type: string
   • Description: The specific role or job title of the target contact.
   • Example: founder, CEO, marketing manager
   • Default: founder
   • Required: Yes

🚀 Invocation Method

The Agent should execute the following independent script to achieve "one command gets results":

# Example
python -u ./scripts/industry_key_contact_radar_api.py "Browser automation" 10 "facebook.com" "founder"

⏳ Running Status Monitoring

Since this task involves automated browser operations, it may take a long time (several minutes). The script will continuously output status logs with timestamps while running (e.g., [14:30:05] Task Status: running). Agent guidelines:

• While waiting for the script to return results, please keep an eye on the terminal output.
• As long as the terminal continues to output new status logs, it means the task is running normally. Do not misjudge it as a deadlock or unresponsiveness.
• If the status remains unchanged for a long time or the script stops outputting without returning a result, only then consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script will parse and print the results directly from the API response. The results include:

• url: Direct link to the contact's public profile
• name: The name of the contact or profile title
• Introduction: A brief introduction or bio description of the contact
• Company: The company or organization associated with the contact

⚠️ Error Handling & Retry

During script execution, if errors occur (such as network fluctuations or task failure), the Agent should follow this logic:

1. Check the output content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. At this point, do not retry, but guide the user to recheck and provide the correct API Key.
   • If the output contains "concurrent" or "too many running tasks" or similar concurrency limit messages, it means the concurrent task limit for the current subscription plan has been reached. Do not retry; guide the user to upgrade their plan. Agent must inform the user:

     "The current task cannot be executed because your BrowserAct account has reached the limit of concurrent tasks. Please go to the BrowserAct Plan Upgrade Page to upgrade your subscription plan and enjoy more concurrent task benefits."

   • If the output does not contain the above error keywords but the task fails (e.g., output starts with Error: or returns empty results), the Agent should automatically try to run the script once more.

2. Retry limit:

   • Automatic retry is limited to once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Lead Generation: Finding public profiles for founders and CEOs in target industries.
2. Talent Acquisition: Sourcing growth leaders and key roles for recruiting.
3. Competitive Analysis: Identifying key personnel in competing organizations.
4. Professional Networking: Gathering profiles for industry connections.
5. Market Intelligence: Collecting URLs and names of industry leaders.
6. Cross-Platform Discovery: Searching target profiles on specific sites like LinkedIn or GitHub.
7. B2B Outreach: Finding decision-makers and extracting contact details.
8. Brand Marketing: Locating marketing managers in a specific field.
9. Sales Prospecting: Building lists of specific job titles across industries.
10. Targeted Job Title Search: Compiling a list of target roles on specific social sites.

LinkedIn — Job Search

keywords + location + filters → paginated job list with full details

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Search LinkedIn job listings with full filter support, extract complete job data with full field coverage.

Prerequisites

• The browser is open and the LinkedIn session is active (logged in). A LinkedIn jobs search page such as https://www.linkedin.com/jobs/search/ must have been visited at least once so the CSRF token cookie is set.

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for LinkedIn has been confirmed in the current session → skip this step.

Otherwise: open https://www.linkedin.com and observe the page:

• User avatar or "Me" menu visible → logged in, continue
• Sign in / Join button visible → not logged in, inform user that LinkedIn login is required first

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It accesses LinkedIn through the user's logged-in browser, only reading data already available to the user. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

API: Search LinkedIn jobs (list page)

eval "$(python scripts/search-jobs.py '{keywords}' '{location}' --count {count} --start {start} --work-type {work_type} --job-type {job_type} --experience {experience} --time-posted {time_posted} --company-ids {company_ids})"

Parameters:

• keywords: job title or search keywords (e.g., software engineer, data analyst)
• location: location name (e.g., United States, New York, San Francisco Bay Area)
• --count: results per API call, default 25, max 100
• --start: pagination offset, default 0. Increment by count for each page
• --work-type: work arrangement filter — 1=On-site, 2=Remote, 3=Hybrid (optional)
• --job-type: contract type filter — F=Full-time, P=Part-time, C=Contract, T=Temporary, I=Internship, V=Volunteer (optional)
• --experience: experience level filter — 1=Internship, 2=Entry, 3=Associate, 4=Mid-Senior, 5=Director (optional)
• --time-posted: recency filter — r86400=24h, r604800=7 days, r2592000=30 days (optional)
• --company-ids: comma-separated LinkedIn company numeric IDs (optional, e.g., 76987811,1441)

Output example:

{
  "total": 36015,
  "start": 0,
  "count": 5,
  "jobs": [
    {
      "id": "4416832078",
      "title": "Lead Frontend Software Engineer",
      "company": "RowsOne",
      "location": "Boca Raton, FL",
      "workType": "Remote",
      "jobUrl": "https://www.linkedin.com/jobs/view/4416832078",
      "companyUrl": "https://www.linkedin.com/company/rowsone"
    }
  ]
}

Error handling: If {"error": true} is returned, check that the browser is still logged in to LinkedIn and navigate to https://www.linkedin.com/jobs/search/ to refresh the session, then retry once.

API: Get full job details

eval "$(python scripts/job-detail.py '{job_id}')"

Parameters:

• job_id: numeric LinkedIn job posting ID (from id field in search results)

Output example:

{
  "id": "4416832078",
  "title": "Lead Frontend Software Engineer",
  "company": "RowsOne",
  "companyUrl": "https://www.linkedin.com/company/rowsone",
  "location": "Boca Raton, FL",
  "workType": "Remote",
  "contractType": "Full-time",
  "experienceLevel": "Mid-Senior level",
  "listedAt": "2026-05-26T16:14:30.000Z",
  "applicantCount": 37,
  "description": "Lead Frontend Engineer (React / Next.js)...",
  "salary": null,
  "jobUrl": "https://www.linkedin.com/jobs/view/4416832078"
}

Error handling: HTTP 404 means job has been removed or ID is invalid. If {"error": true, "message": "HTTP 403"}, the LinkedIn session may have expired — navigate back to LinkedIn and verify login, then retry.

Composite: Full job extraction (search list + detail for each job)

For complete output with all fields (description, contract type, experience level, posted date):

1. Run search component to collect job IDs and basic info
2. For each job ID, run the detail component
3. Merge results by job ID

Batch script template (bash):

#!/bin/bash
SESSION="fb_explore"
KEYWORDS="software engineer"
LOCATION="United States"
TOTAL_ROWS=50
COUNT=25
OUTPUT_FILE="output/jobs.jsonl"

offset=0
collected=0
while [ $collected -lt $TOTAL_ROWS ]; do
  batch_count=$((TOTAL_ROWS - collected))
  [ $batch_count -gt $COUNT ] && batch_count=$COUNT

  result=$(browser-act --session $SESSION eval "$(python scripts/search-jobs.py "$KEYWORDS" "$LOCATION" --count $batch_count --start $offset)")
  echo "$result" | python -c "
import json, sys
data = json.loads(sys.stdin.read())
for job in data.get('jobs', []):
    print(json.dumps(job))
" >> output/jobs_basic.jsonl

  job_ids=$(echo "$result" | python -c "import json,sys; [print(j['id']) for j in json.loads(sys.stdin.read()).get('jobs',[])]")
  for job_id in $job_ids; do
    detail=$(browser-act --session $SESSION eval "$(python scripts/job-detail.py $job_id)")
    echo "$detail" >> $OUTPUT_FILE
    sleep 1
  done

  page_count=$(echo "$result" | python -c "import json,sys; print(json.loads(sys.stdin.read()).get('count',0))")
  [ "$page_count" -eq 0 ] && break
  collected=$((collected + page_count))
  offset=$((offset + page_count))
  sleep 2
done
echo "Done. Collected $collected jobs."

Note: Add sleep 1 between detail calls to avoid rate limiting. For large batches (>200 jobs), use multiple browser sessions in parallel — each session counts independently toward rate limits.

Enum Parameters

Filter values are hardcoded in scripts; no dynamic enumeration needed.

Work type (--work-type): 1=On-site, 2=Remote, 3=Hybrid

Contract type (--job-type): F=Full-time, P=Part-time, C=Contract, T=Temporary, I=Internship, V=Volunteer

Experience level (--experience): 1=Internship, 2=Entry level, 3=Associate, 4=Mid-Senior level, 5=Director

Time posted (--time-posted): r86400=Past 24 hours, r604800=Past week, r2592000=Past month

Pagination

API Pagination: parameter --start, type: page-offset, start value: 0. Next page: increment by --count value. Termination: when count in response is 0, or start >= total, or start >= rows target.

LinkedIn typically returns results up to start=1000 maximum regardless of total.

Success Criteria

result count >= 1 and jobs[0].id is non-null

Known Limitations

• LinkedIn limits accessible search results to approximately the first 1000 jobs per query even when total shows a higher number
• experienceLevel may be null for many postings — companies do not always fill in this field
• salary is null for most postings; LinkedIn only shows salary when the employer explicitly provides it
• Rate limiting: sustained rapid requests (e.g., >100 detail calls without sleep) may trigger temporary blocks. Add sleep 1 between detail calls
• Login required: unlike public job boards, LinkedIn's Voyager API requires an authenticated session. The CSRF token is derived from the JSESSIONID cookie set at login

Execution Efficiency

• Batch orchestration: write a bash loop iterating over job IDs serially; do not parallelize within one browser. For higher throughput, use multiple stealth browsers with separate sessions
• Test before batch: run with --count 3 first to confirm the script runs correctly before scaling up
• Error resumption: append results to .jsonl file line-by-line so the job can resume from a specific offset on failure
• Search only for large volumes: for >500 jobs where full description is not needed, use the search component alone — it returns title, company, location, work type, and URLs without per-job detail calls

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/linkedin-job-search-linkedin-jobs-search.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Product Hunt — Launch Data Extraction

Input: date/period parameters → Output: structured product launch data with maker profiles and website contact info

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract complete product launch data from Product Hunt leaderboard pages, enriched with maker profile information and product website contact details.

Prerequisites

• Browser session is open and can access producthunt.com
• Cloudflare challenge may appear on first visit; use solve-captcha or wait for auto-pass

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Cloudflare Verification

Product Hunt uses Cloudflare protection. On first navigation:

1. Navigate to target URL
2. If page title shows "Just a moment..." → wait stable --timeout 15000 then check title again
3. If still blocked → solve-captcha
4. Verify page loaded: title should contain "Product Hunt" or "Best of Product Hunt"

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py)". $(...) is bash syntax; use the bash tool for execution.

DOM: Extract product list from leaderboard page

Navigate to the target leaderboard URL first, then extract:

eval "$(python scripts/extract-leaderboard.py)"

URL patterns (navigate to the appropriate one before extraction):

• Daily: https://www.producthunt.com/leaderboard/daily/{YYYY}/{M}/{DD}/all
• Weekly: https://www.producthunt.com/leaderboard/weekly/{YYYY}/{week-number}/all
• Monthly: https://www.producthunt.com/leaderboard/monthly/{YYYY}/{M}/all
• Yearly: https://www.producthunt.com/leaderboard/yearly/{YYYY}/all

Replace all with featured for featured-only products.

Output example:

[
  {
    "rank": 1,
    "name": "Product Name",
    "tagline": "Short product description",
    "categories": ["Productivity", "AI"],
    "thumbnail": "https://ph-files.imgix.net/...",
    "upvotes": 135,
    "comments": 42,
    "url": "https://www.producthunt.com/products/product-slug",
    "slug": "product-slug"
  }
]

DOM: Extract launch detail page

Navigate to the launch page URL first (https://www.producthunt.com/products/{slug}/launches/{launch-slug}), then extract:

eval "$(python scripts/extract-launch-detail.py)"

To find the launch URL from a product page: navigate to https://www.producthunt.com/products/{slug} and look for links matching /products/{slug}/launches/{launch-slug}.

Output example:

{
  "name": "Product Name",
  "tagline": "Short product tagline",
  "description": "Full product description from OG meta",
  "categories": ["Productivity", "Social Media"],
  "images": ["https://ph-files.imgix.net/gallery1.png", "https://ph-files.imgix.net/gallery2.png"],
  "websiteUrl": "https://product-website.com/?ref=producthunt",
  "upvotes": 135,
  "launchDate": "2025-05-27T07:26:33-07:00",
  "makers": [{"href": "/@username", "name": "Maker Name"}],
  "ogImage": "https://ph-files.imgix.net/og-image.png"
}

DOM: Extract maker profile

Navigate to maker profile URL (https://www.producthunt.com/@{username}), then extract:

eval "$(python scripts/extract-maker-profile.py)"

Output example:

{
  "name": "Maker Name",
  "slug": "@username",
  "headline": "Creating SaaS Products",
  "aboutText": "Bio text about the maker",
  "links": ["https://twitter.com/username", "https://linkedin.com/in/username"],
  "followers": 22,
  "url": "https://www.producthunt.com/@username"
}

DOM: Extract website email and content

Navigate to the product website URL, wait for load, then extract:

eval "$(python scripts/extract-website-content.py)"

Alternatively, use stealth-extract for faster extraction without a browser session: stealth-extract {website-url} --content-type markdown then parse the markdown for email patterns.

Output example:

{
  "title": "Product Website Title",
  "url": "https://product-website.com",
  "email": "contact@product-website.com",
  "allEmails": ["contact@product-website.com", "support@product-website.com"],
  "websiteRawText": "Full visible text content of the website..."
}

Composite: Full product extraction (leaderboard + detail + maker + website)

Complete pipeline replicating the full Product Hunt scraper workflow:

1. Navigate to leaderboard page → wait stable → eval "$(python scripts/extract-leaderboard.py)"
2. For each product from step 1: a. Navigate to https://www.producthunt.com/products/{slug} → find launch link → navigate to launch page b. wait stable → eval "$(python scripts/extract-launch-detail.py)" → get full details + maker links + website URL
3. (Optional, if scrapeMakers is enabled) For each unique maker from step 2: a. Navigate to https://www.producthunt.com/{maker.href} → wait stable → eval "$(python scripts/extract-maker-profile.py)"
4. (Optional, if scrapeWebsite is enabled) For each product website URL from step 2: a. Navigate to website URL → wait stable → eval "$(python scripts/extract-website-content.py)"
5. Merge all data by product slug

Final output example per product:

{
  "date": "2026-06-10T00:00:00Z",
  "launchDate": "2026-06-10T07:01:04Z",
  "url": "https://www.producthunt.com/products/product-slug",
  "name": "Product Name",
  "shortDescription": "Short tagline",
  "description": "Full description text",
  "categories": ["Productivity", "AI"],
  "maker": {
    "makerHref": "https://www.producthunt.com/@username",
    "name": "Maker Name",
    "slug": "@username",
    "url": "https://www.producthunt.com/@username",
    "links": ["https://twitter.com/maker", "https://linkedin.com/in/maker"],
    "aboutText": "Maker bio text"
  },
  "websiteUrl": "https://product-website.com",
  "images": ["https://ph-files.imgix.net/image1.png"],
  "upvotes": 135,
  "website": {
    "title": "Product Website",
    "url": "https://product-website.com",
    "email": "hello@product-website.com",
    "websiteRawText": "Full page text content..."
  }
}

Pagination

No pagination required for daily/weekly leaderboard: All products for a given day load on a single page (typically 15-50 products per day). No infinite scroll or "load more" button exists.

Yearly leaderboard: May contain many products. Apply topNProducts filter to limit. All visible products are rendered on the single page.

Success Criteria

• result count >= 1 (at least one product extracted from leaderboard)
• Core fields non-null: name, tagline, upvotes, url present for every product
• Data consistency: extracted product names match what is displayed on the page
• When detail enrichment is performed: websiteUrl or maker present for enriched items

Known Limitations

• Cloudflare protection requires initial challenge pass; may need solve-captcha on first visit
• No public API available; Product Hunt only accepts persisted GraphQL queries. All data must be extracted via DOM
• Rate limiting: rapid sequential page navigations may trigger Cloudflare blocks. Add 2-3 second delays between product detail page visits
• The /all URL path (used by older scrapers) now returns 404; use /leaderboard/daily/ path instead
• Product detail pages may vary in structure for older launches vs newer ones
• Website email extraction depends on email being visible in page text or HTML; mailto links and contact forms with obfuscated emails will not be captured

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser. Add 2-3 second delays between navigations to avoid Cloudflare blocks. To increase throughput, open multiple stealth browser sessions and distribute work across them
• Test before batch execution: After writing a batch script, first test with 1-2 items to verify the script runs correctly; only then run the full batch
• Reduce redundant pre-operations: The leaderboard extraction gives all basic data in one pass; only visit detail pages when full description, images, or maker info are needed
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over
• Skip website extraction when not needed: Website content extraction is the slowest step (external site navigation). Only enable when email/content data is specifically required

Experience Notes

Path: browser-act-skill-forge-memories/producthunt-scraper-producthunt-launches.memory.md (working directory is determined by the Agent running the Skill)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file.

Social Media Finder Skill

📖 Brief

This skill automates the discovery of social media accounts associated with individuals, brands, or businesses across multiple platforms. By providing a name, it searches across Facebook, Twitter, Instagram, LinkedIn, TikTok, and more — extracting profile URLs, usernames, follower counts, and bio snippets in one go. The results are returned as a downloadable CSV file.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key yet, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

The agent should configure the following parameter based on user requirements:

1. People_Name
   • Required: Yes
   • Type: string
   • Description: The name of the person or brand to search for. Use + between words for better results.
   • Example: John+Smith, Tesla, Bill+Gates
   • Default: Mike+Smith

🚀 Invocation Method

Agent should execute the following command to invoke the skill:

# Example invocation
python -u ./scripts/social_media_finder.py "John+Smith"

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take several minutes. The script outputs timestamped status logs continuously (e.g., [14:30:05] Task Status: running). Agent guidelines:

• Monitor the terminal output while waiting.
• As long as new status logs appear, the task is running normally; do not misjudge it as frozen.
• Only consider triggering retry if the status remains unchanged for a long time or output stops without a final result.

📊 Data Output

Upon successful execution, the script retrieves a JSON object containing a files array with a download link to a CSV file. The CSV includes the following structured profile data:

• Platform name: The social media platform (e.g., LinkedIn, Twitter).
• Followers count: The number of followers.
• Page title: The title of the profile page.
• Description snippet: The bio or description snippet.
• Profile URL: The direct link to the social media profile.

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output contains "concurrent" or "too many running tasks", it means the concurrent task limit has been reached. Do not retry; guide the user to upgrade their plan. Agent must inform the user:

     "The current task cannot be executed because your BrowserAct account has reached the concurrent task limit. Please visit the BrowserAct Plan Upgrade Page to upgrade your plan."

   • If the output does not contain the above error keywords but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error to the user.

🌟 Typical Use Cases

1. Background Research: Find social media profiles of job candidates or new connections.
2. Lead Generation: Gather contact information and social profiles for sales prospects.
3. Competitive Intelligence: Discover the social media footprint of competitors.
4. Influencer Discovery: Locate content creators across various platforms.
5. Brand Monitoring: Check a brand's presence across different social networks.
6. Reconnecting: Find old friends or colleagues online.
7. Identity Verification: Cross-reference profiles of people met online.
8. Customer Support: Find official brand accounts for reaching out.
9. Networking: Research event attendees or potential business partners.
10. Digital Footprint Audit: Monitor one's own public social media presence.

YouTube — Channel Business Email & Contact Extractor

Input: a YouTube channel id (UCxxx), handle (@name), or channel URL. Output: structured channel metadata (id, name, counters, country, joined date, external links classified by kind) plus the channel's business email if it can be found on the About page description or on any linked personal/business website.

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract a YouTube channel's business inquiry email together with the channel's About metadata and outbound link list, prioritising what is already publicly displayed on the channel's About page; if no email is visible there, walk the channel's own outbound links (personal site, business site, link aggregator) to recover an email that the creator publishes on a site they control.

Prerequisites

• Target page is already open in the browser: https://www.youtube.com/{@handle|channel/UCxxx}/about
• No login is required for description and link extraction; running the browser while signed in to YouTube does not change what is returned by this Skill.

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory; each .py script prints a self-contained JS snippet to stdout. To execute that snippet in the browser, hand it to browser-act's eval subcommand via bash command substitution: browser-act --session <session-name> eval "$(python scripts/xxx.py {params})". The outer $(...) is bash syntax; use the bash tool to run it. Pure-Python scripts that already emit a JSON result on stdout (e.g. normalize-channel-input.py, extract-emails-from-text.py) are invoked directly: python scripts/xxx.py {params} — do NOT wrap them in browser-act eval.

API: normalize a channel input into a canonical /about URL

python scripts/normalize-channel-input.py '{channel-input}'

Parameters:

• {channel-input}: a channel id (UCxxxxxxxxxxxxxxxxxxxxxx, 24 chars starting with UC), a handle (@name), a bare handle (name), or any channel URL (https://www.youtube.com/@name, https://www.youtube.com/channel/UCxxx, https://www.youtube.com/c/legacy, https://www.youtube.com/user/legacy). Trailing path segments and querystrings are stripped.

Output example:

{
  "about_url": "https://www.youtube.com/@example/about",   // canonical URL to navigate to before extraction
  "input_type": "handle",                                  // one of: handle | channel_id | channel_url | legacy_custom_url | bare_handle | raw_url
  "value": "@example"                                      // normalised value (handle with @ prefix, or channel id, or raw url)
}

On unrecognised input the script returns {"error": true, "message": "unrecognized input: ..."}.

DOM: extract channel About metadata + emails from description

Run from the channel's /about view (after navigate + wait stable). Reads window.ytInitialData.aboutChannelViewModel and pulls the description, full counter set, channel id, canonical URL, the resolved outbound links (with the YouTube redirect wrapper unwrapped), and any email addresses already present in the description text.

Extract: browser-act --session <session-name> eval "$(python scripts/extract-channel-about.py)"

Output example:

{
  "channel_id": "UCxxxxxxxxxxxxxxxxxxxxxx",                // YouTube channel id (UCxxx)
  "channel_name": "Channel Display Name",                  // display name from channel metadata
  "canonical_channel_url": "http://www.youtube.com/@example", // canonical channel URL reported by YouTube
  "description": "Short bio line ...\n\nbusiness@example.com\n\nCity", // full About description text
  "country": "United States",                              // country shown on the About panel (localised text)
  "subscriber_count_text": "21M subscribers",              // raw display text (locale-formatted)
  "view_count_text": "5,455,901,172 views",                // raw display text (locale-formatted)
  "video_count_text": "1,831 videos",                      // raw display text (locale-formatted)
  "joined_date_text": "Joined Mar 21, 2008",               // raw display text (locale-formatted)
  "has_business_email_reveal_button": true,                // whether the "View email address" button is exposed on the page
  "bypass_business_email_captcha": false,                  // whether the current viewer can skip the email reveal captcha
  "emails_in_description": ["business@example.com"],       // emails matched in the description text; empty array when none
  "links": [                                               // outbound links with YouTube /redirect wrappers unwrapped
    {"title": "Twitter", "display": "twitter.com/example", "url": "http://twitter.com/example"}
  ]
}

On failure (page is not a channel /about view, ytInitialData missing, or aboutChannelViewModel not present), the script returns {"error": true, "message": "..."}.

API: extract emails from arbitrary text + classify a URL by kind

python scripts/extract-emails-from-text.py [--text '{text}' | --text-file {path}] [--classify-url '{url}']

Parameters:

• --text: raw text to scan for email addresses (plain, markdown, or HTML)
• --text-file: path to a UTF-8 text file to scan for email addresses
• --classify-url: URL to classify into social, link_aggregator, or personal_or_business (used to decide which outbound links are worth fetching during email recovery)

Output example (with --text and --classify-url):

{
  "classification": {
    "kind": "personal_or_business",          // social | link_aggregator | personal_or_business | invalid
    "domain": "example.com",
    "url": "https://example.com/"
  },
  "emails": ["info@example.com"]             // deduplicated, image/file extensions filtered out, obfuscated forms like name [at] brand [dot] io are recovered
}

Without any argument the script returns {"error": true, "message": "provide --text, --text-file, or --classify-url"}.

Composite: end-to-end channel-to-business-email lookup

Cross-page composite: input normalisation → navigation → About extraction → optional follow of one or more non-social outbound links to recover an email when the description had none.

Before the loop, ensure a scratch directory exists for the markdown dumps used in step 6: mkdir -p tmp (run once per batch, not per channel).

1. Normalise the input: python scripts/normalize-channel-input.py '{channel-input}' — read about_url from stdout.
2. Navigate the browser session to that URL and wait for it to render: browser-act --session <session-name> navigate {about_url} → browser-act --session <session-name> wait stable --timeout 12000 (a Timed out waiting for page readiness here is recoverable — proceed if the URL and title reflect the channel; check Known Limitations).
3. Pull the channel About payload: browser-act --session <session-name> eval "$(python scripts/extract-channel-about.py)"
4. If emails_in_description is non-empty, set business_email = emails_in_description[0] and email_source = "description". Stop and emit the result.
5. Otherwise classify each link to decide which outbound URLs to walk; skip social links (Twitter/Instagram/TikTok/Facebook/etc. — they require login and are unreliable to scrape): For each link.url in links: python scripts/extract-emails-from-text.py --classify-url '{link.url}'
6. For every link classified as personal_or_business or link_aggregator, fetch its rendered text and look for an email; stop at the first match:
   • browser-act stealth-extract '{link.url}' --content-type markdown > tmp/site.md
   • python scripts/extract-emails-from-text.py --text-file tmp/site.md
   • If the result has non-empty emails, set business_email = emails[0], email_source = "linked_website:{classification.domain}" and stop.
   • Order of attempts: personal_or_business first (most likely to host the creator's own contact page), then link_aggregator (linktr.ee / beacons.ai style pages that may surface a contact email).
7. If no email is recovered, emit business_email = null, email_source = null, status = "no_email_found".

Final emitted record (one per channel input):

{
  "input": "@example",                                       // original input as provided
  "channel_id": "UCxxxxxxxxxxxxxxxxxxxxxx",                  // resolved YouTube channel id (UCxxx)
  "channel_name": "Channel Display Name",                    // display name from channel metadata
  "channel_url": "https://www.youtube.com/@example",         // canonical channel URL
  "business_email": "business@example.com",                  // recovered email, or null when none was found
  "email_source": "description",                             // description | linked_website:{domain} | link_aggregator:{domain} | null
  "country": "United States",                                // localised country text
  "subscriber_count_text": "1.2M subscribers",               // raw display text
  "view_count_text": "123,456,789 views",                    // raw display text
  "video_count_text": "987 videos",                          // raw display text
  "joined_date_text": "Joined Jan 1, 2015",                  // raw display text
  "has_business_email_reveal_button": true,                  // whether the "View email address" gate is offered on the channel
  "links": [                                                 // outbound links with classification appended
    {"title": "Instagram", "display": "instagram.com/example", "url": "http://instagram.com/example", "kind": "social", "domain": "instagram.com"}
  ],
  "status": "ok"                                             // ok | no_email_found | error
}

On any unrecoverable error (input normalisation failed, navigation failed, extraction returned error: true), emit status = "error" together with the failing step name in error_step and the error message in error_message.

Known Limitations

• The "View email address" button on /about is gated by reCAPTCHA and a YouTube server-side validation that rejects programmatically-solved captcha tokens (the call to youtubei/v1/channel/reveal_business_email returns HTTP 400 INVALID_ARGUMENT even when a captcha solver returns success). Emails that only exist behind that reveal button cannot be recovered by this Skill — only emails published in the description text or on a creator-controlled outbound site are returned.
• wait stable on YouTube /about pages frequently exceeds the default 30s due to long-lived media streams; the extraction is still valid as long as ytInitialData.aboutChannelViewModel is present (verified after the timeout). Treat a single Timed out waiting for page readiness as recoverable and run the extraction script anyway.
• country, subscriber_count_text, view_count_text, video_count_text, joined_date_text are returned verbatim in the locale that the underlying browser renders (e.g. an en locale yields 21M subscribers while a zh-CN locale yields its own localised string with native digit grouping and the localised word for "subscribers"). Downstream consumers must do the parsing.
• Description-based email matching only finds emails that the creator typed into the About description. Many large creators (e.g. channels with has_business_email_reveal_button = true and an empty emails_in_description) keep their email behind the captcha-gated button and will therefore not be recovered.
• Outbound link walking is limited to one HTTP fetch per non-social link via stealth-extract (no JS interaction, no crawling deeper than the landing page). Sites that hide their email behind a contact form or a JS-rendered modal that does not appear in the initial markdown will yield no email.
• Image asset URLs whose path contains @ (e.g. image@2x.png) and YouTube video URLs containing @handle are deliberately filtered out of the email match results; the trade-off is that very unusual emails ending in .png/.jpg/.gif/.webp/.svg/.ico would also be rejected (not encountered in practice).

Execution Efficiency

• Batch orchestration: Write a bash script to loop through channel inputs serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session.
• Test before batch execution: After writing a batch script, you must first test with 1-2 channels to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly.
• Reduce redundant pre-operations: Keep one browser session open across the whole batch (one browser open then many navigates); avoid reopening the browser between channels.
• Error resumption: Save results item by item during batch processing (append one JSON line per channel to an output file); on failure, resume from the breakpoint rather than starting over.
• Skip social link walking when not needed: The first match wins; once emails_in_description is non-empty, do not fetch any outbound link.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/youtube-channel-business-email-youtube-channel-business-email.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Google Image API Automation Skill

📖 Introduction

This skill provides users with one-click image data extraction directly from Google Images using the BrowserAct Google Image API template. It allows you to search with keywords, set country and language, control scroll depth and result limits, returning clean, structured image metadata directly via API.

✨ Features

1. No hallucinations, ensuring stable and accurate data extraction: Pre-set workflows avoid generative AI hallucinations.
2. No CAPTCHA issues: No need to deal with reCAPTCHA or other verification challenges.
3. No IP restrictions or geo-blocking: No need to handle regional IP limitations.
4. Agile execution speed: Faster task execution compared to pure AI-driven browser automation solutions.
5. High cost-effectiveness: Significantly reduces data acquisition costs compared to AI solutions that consume a large number of tokens.

🔑 API Key Guide

Before running, you must check the BROWSERACT_API_KEY environment variable. If it is not set, do not take any further action; you should request and wait for the user to provide it collaboratively. The Agent must inform the user at this point:

"Since you haven't configured the BrowserAct API Key yet, please go to the BrowserAct Console to get your Key first."

🛠️ Input Parameters

The Agent should flexibly configure the following parameters according to user needs when calling the script:

1. KeyWords (Search keywords)

   • Type: string
   • Description: Search keywords used on Google Images.
   • Example: flower, ai agent, tesla

2. Country (Country or region bias)

   • Type: string
   • Description: Country or region bias for results.
   • Supported values: us, gb, ca, au, de, fr, es, jp, kr
   • Default: us

3. Language (UI language)

   • Type: string
   • Description: UI language for the Google Images session and returned text.
   • Supported values: en, zh-CN, zh-TW, ja, ko, fr, de, es
   • Default: en

4. Scroll_count (Number of scroll actions)

   • Type: number
   • Description: Number of scroll actions to load more image results.
   • Default: 5

5. Datelimit (Maximum items)

   • Type: number
   • Description: Maximum number of items to extract from the results list.
   • Default: 50

🚀 Invocation (Recommended)

The Agent should execute the following independent script to achieve "results with one command":

# Example invocation
python -u ./scripts/google_image_api.py "KeyWords" "Country" "Language" Scroll_count Datelimit

⏳ Execution Status Monitoring

Since this task involves automated browser operations, it may take a considerable amount of time (several minutes). The script will continuously output status logs with timestamps while running (e.g., [14:30:05] Task Status: running). Agent Notice:

• While waiting for the script to return results, please keep an eye on the terminal output.
• As long as the terminal is outputting new status logs, it means the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• If the status remains unchanged for a long time or the script stops outputting without returning a result, then consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script will parse and print the results directly from the API response. The results include:

• is_product: Whether the result is detected as a product-style listing
• link: Click-through URL associated with the result
• title: Image result title or caption text
• source_logo: Source site logo URL
• source: Source site name shown in results
• related_content_id: Google Images related content identifier
• thumbnail: Thumbnail image URL
• index: Result index in the list

⚠️ Error Handling & Retry

During the execution of the script, if an error occurs (such as network fluctuation or task failure), the Agent should follow this logic:

1. Check the output:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. In this case, do not retry; guide the user to check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task execution fails (for example, the output starts with Error: or the result is empty), the Agent should automatically try executing the script one more time.

2. Retry limit:

   • Automatic retry is limited to once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Visual Content Sourcing: Finding specific imagery for creative research and design content.
2. Competitor Asset Monitoring: Scanning Google Images for competitor product styles and logos.
3. Market Visual Research: Building datasets of product listings across various countries.
4. Localized Image Trends: Tracking what images appear for specific terms in Japan (jp) or France (fr).
5. E-commerce Discovery: Extracting click-through links to track down where products are sold.
6. Data Enrichment: Fetching thumbnails and high-level titles associated with keywords.
7. Brand Tracking: Finding instances of specific brands appearing as image results.
8. SEO Keyword Visualization: Checking the visual results that rank for chosen SEO keywords.
9. Automated Content Aggregation: Delivering daily list-level visual metadata for specific topics.
10. Global Image Search: Finding images related to global events or personalities in their native languages.

Google News Automation Skill

📖 Introduction

This skill provides a one-stop news collection service using BrowserAct's Google News API template. It directly extracts structured news results from Google News, including headlines, sources, publication times, and article links. Simply input search keywords and time filters to get clean, usable news data.

✨ Features

1. No Hallucinations: Pre-set workflows avoid AI generative hallucinations, ensuring stable and precise data extraction.
2. No Captcha Issues: No need to handle reCAPTCHA or other verification challenges.
3. No IP Restrictions: No need to handle regional IP restrictions or geofencing.
4. Faster Execution: Tasks execute faster compared to pure AI-driven browser automation solutions.
5. Cost-Effective: Significantly lowers data acquisition costs compared to high-token-consuming AI solutions.

🔑 API Key Setup

Before running, check the BROWSERACT_API_KEY environment variable. If not set, do not take other measures; ask and wait for the user to provide it. Agent must inform the user:

"Since you haven't configured the BrowserAct API Key, please visit the BrowserAct Console to get your Key."

🛠️ Input Parameters

Agent should flexibly configure the following parameters based on user needs:

1. Search_Keywords (Search Keywords)

   • Type: string
   • Description: The content to search on Google News. Can be company names, industry terms, competitor names, etc.
   • Example: AI Startup, Tesla, SpaceX

2. Publish_date (Time Range)

   • Type: string
   • Description: Filter news by publication time.
   • Options:
     • any time: No restriction
     • past hours: Past few hours (breaking news)
     • past 24 hours: Past 24 hours (daily monitoring)
     • past week: Past week (short-term trends)
     • past year: Past year (long-term research)
   • Default: past week

3. Datelimit (Max Items)

   • Type: number
   • Description: Maximum number of news items to extract per task.
   • Default: 30
   • Suggestion: Use 10-30 for real-time monitoring; use larger values for deep research.

🚀 Usage

Agent should use the following independent script to achieve "one-line command result":

# Example
python -u ./scripts/google_news_api.py "Search Keywords" "Publish Date" Quantity

⏳ Execution Monitoring

Since this task involves automated browser operations, it may take some time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running). Agent Instructions:

• While waiting for the script result, keep monitoring the terminal output.
• As long as the terminal is outputting new status logs, the task is running normally; do not mistake it for a deadlock or unresponsiveness.
• Only if the status remains unchanged for a long time or the script stops outputting without returning a result should you consider triggering the retry mechanism.

📊 Data Output

After successful execution, the script parses and prints results directly from the API response. Results include:

• headline: News title
• source: Publisher/Source
• news_link: Article URL
• published_time: Publication time
• author: Author (if available)

⚠️ Error Handling & Retry

If an error occurs during script execution (e.g., network fluctuations or task failure), the Agent should follow this logic:

1. Check Output Content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. Do not retry; guide the user to re-check and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task failed (e.g., output starts with Error: or returns empty results), the Agent should automatically try to re-execute the script once.

2. Retry Limit:

   • Automatic retry is limited to one time. If the second attempt fails, stop retrying and report the specific error information to the user.

🌟 Typical Use Cases

1. Industry Trend Tracking: Find the latest developments in fields like "Low-altitude economy" or "Generative AI".
2. PR Monitoring: Monitor media exposure of a specific brand or company over the past 24 hours.
3. Competitor Intelligence: Collect information on new products or marketing activities from competitors over the past week.
4. Market Research: Get popular reports on specific keywords across different time dimensions.
5. Figure Tracking: Retrieve the latest news reports on industry leaders or public figures.
6. Daily News Summary: Automatically extract and summarize daily news in specific domains.
7. Global Breaking News: Get real-time updates on major global events.
8. Structured Data Extraction: Extract structured information like headlines, sources, and links for analysis.
9. Media Exposure Analysis: Evaluate the propagation heat of a project or event in mainstream news media.
10. Long-term Research: Retrieve all in-depth reports on a specific technical topic from the past year.

Google — Search SERP Extraction

Search keyword + parameters → structured SERP data (organic results, ads, related queries, PAA, AI Overview)

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Extract all visible content from a Google Search results page: organic listings, paid ads, related searches, People Also Ask, AI Overview, and total result count.

Prerequisites

• Target page is already open in the browser: https://www.google.com/search?q={query}

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

DOM: Google Search SERP (data extraction)

Parameters are injected via URL navigation; data is extracted from the server-rendered HTML page:

1. navigate https://www.google.com/search?q={query}&num={num}&hl={lang}&gl={country}&start={start}
2. wait stable
3. eval "$(python scripts/serp-extract.py)"

URL parameters:

• q: Search query (required)
• num: Results per page — 10 (default), 20, 50, 100
• hl: Interface language code — e.g., en, zh-CN, fr, de (omit for browser default)
• gl: Country targeting code — e.g., us, gb, de, cn (omit for browser default)
• start: Pagination offset — 0 for page 1, 10 for page 2 (when num=10); formula: (page - 1) * num

Error handling: If extraction returns {"error": true, "message": "captcha required"}, the session is blocked by Google — switch to a browser with a US rotating proxy and retry. If "No search results found" is returned, run screenshot to verify the page loaded correctly before retrying.

Output example:

{
  "searchQuery": {
    "term": "machine learning",
    "url": "https://www.google.com/search?q=machine+learning",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "www.google.com",
    "countryCode": "US",
    "languageCode": "en"
  },
  "resultsTotal": "14900000000",
  "organicResults": [
    {
      "position": 1,
      "type": "organic",
      "title": "Machine learning - Wikipedia",
      "url": "https://en.wikipedia.org/wiki/Machine_learning",
      "displayedUrl": "en.wikipedia.org › wiki › Machine_learning",
      "description": "Machine learning (ML) is a field of study in artificial intelligence...",
      "emphasizedKeywords": ["machine learning", "ML"],
      "siteLinks": [
        {"title": "Supervised learning", "url": "https://en.wikipedia.org/wiki/Supervised_learning"}
      ]
    }
  ],
  "paidResults": [
    {
      "adPosition": 1,
      "type": "paid",
      "title": "Learn Machine Learning Online",
      "url": "https://example.com/ml-course",
      "displayedUrl": "example.com",
      "description": null,
      "siteLinks": []
    }
  ],
  "relatedQueries": [
    {"title": "machine learning examples", "url": "https://www.google.com/search?q=machine+learning+examples"}
  ],
  "peopleAlsoAsk": [
    {"question": "What is machine learning used for?"}
  ],
  "aiOverview": null
}

Field notes:

• resultsTotal: total result count string (commas removed), null when stat bar is absent
• organicResults[*].emphasizedKeywords: bold/italic terms in the description, empty array when none
• organicResults[*].siteLinks: sub-links shown under some results, empty array when none
• paidResults[*].description: ad description text, null when the advertiser omits it
• aiOverview: AI Overview paragraph text joined with spaces, null when absent or unavailable

Pagination

URL Pagination: URL pattern https://www.google.com/search?q={query}&num={num}&start={(page-1)*num}. Increment start by num for each subsequent page. Termination: organicResults array is empty, or start exceeds the desired page count.

Success Criteria

organicResults.length >= 1 and searchQuery.term matches the requested keyword.

Known Limitations

• AI Overview unreliable in stealth sessions: Google rarely serves AI Overview to automated browsers. aiOverview will be null in most sessions; it only populates when Google serves it without login or cookie context.
• Paid ad descriptions often null: Many ads omit a description block — paidResults[*].description returns null for those. This reflects the advertiser's choice, not an extraction failure.
• Google anti-bot detection: Stealth browsers may be redirected to a CAPTCHA (/sorry/ page). Use a browser session with a US rotating proxy to reduce blocks. Solve any CAPTCHA manually via remote-assist if needed.
• Related queries load asynchronously: relatedQueries requires wait stable after navigation; results may be empty if the page has not fully settled.

Execution Efficiency

• Batch orchestration: Write a bash script to loop through keywords serially within one browser session; add a 2–5 second delay between requests to avoid triggering rate limits.
• Test before batch execution: After writing a batch script, test with 1–2 keywords first to verify it runs correctly; only then run the full batch.
• Reduce redundant pre-operations: Reuse the same browser session across multiple keywords — navigate directly to each search URL without returning to the homepage.
• Error resumption: Save results keyword by keyword; on CAPTCHA or failure, resume from the breakpoint rather than starting over.
• Multi-session parallelism: To increase throughput, open multiple stealth browser sessions (each with its own proxy fingerprint) and distribute keywords across them.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/google-search-scraper-google-search-serp.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Web Search Scraper API Skill

📖 Introduction

This skill provides users with a one-stop web page extraction service through the BrowserAct Web Search Scraper API template. It can directly extract structured markdown content from any given URL. By simply inputting the target URL, you can get clean and usable markdown data.

✨ Features

1. No hallucinations, ensuring stable and precise data extraction: Pre-set workflows avoid AI generative hallucinations.
2. No human-machine verification issues: No need to deal with reCAPTCHA or other verification challenges.
3. No IP access restrictions or geofencing: No need to handle regional IP limitations.
4. More agile execution speed: Compared to purely AI-driven browser automation solutions, task execution is faster.
5. Extremely high cost-effectiveness: Compared to AI solutions that consume a lot of Tokens, it can significantly reduce the cost of data acquisition.

🔑 API Key Guidance Process

Before running, you must check the BROWSERACT_API_KEY environment variable. If it is not set, do not take other actions first; you should ask and wait for the user to provide it cooperatively. The Agent must inform the user at this time:

"Since you have not configured the BrowserAct API Key, please go to the BrowserAct Console first to get your Key."

🛠️ Input Parameters Details

Agent should flexibly configure the following parameters based on user needs when calling the script:

1. target_url
   • Type: string
   • Description: The website URL to extract content from. Supports any HTTP/HTTPS URL.
   • Example: https://www.browseract.com

🚀 Invocation Method (Recommended)

Agent should execute the following independent script to achieve "one command gets the result":

# Example invocation
python -u ./scripts/web_search_scraper_api.py "target_url"

⏳ Execution Status Monitoring

Since the task involves automated browser operations, it may take a long time (several minutes). The script will continuously output status logs with timestamps (e.g., [14:30:05] Task Status: running) while running. Notice for Agent:

• While waiting for the script to return results, please keep paying attention to the terminal output.
• As long as the terminal is still outputting new status logs, it means the task is running normally. Do not misjudge it as a deadlock or unresponsiveness.
• If the status remains unchanged for a long time or the script stops outputting and no result is returned, the retry mechanism can be triggered.

📊 Data Output Description

Upon successful execution, the script will directly parse and print the result from the API response. The result contains:

• content: The complete markdown content of the webpage.

⚠️ Error Handling & Retry Mechanism

During the execution of the script, if an error occurs (such as network fluctuation or task failure), the Agent should follow this logic:

1. Check the output content:

   • If the output contains "Invalid authorization", it means the API Key is invalid or expired. At this time, do not retry, and you should guide the user to recheck and provide the correct API Key.
   • If the output does not contain "Invalid authorization" but the task execution fails (for example, the output starts with Error: or the returned result is empty), the Agent should automatically try to re-execute the script once.

2. Retry limit:

   • Automatic retry is limited to once. If the second attempt still fails, stop retrying and report the specific error message to the user.

🌟 Typical Use Cases

1. Article Extraction: Scrape the main content of a news article link into markdown.
2. Blog Post Parsing: Download the readable text from a target blog post URL.
3. Webpage to Markdown: Convert any given website URL into clean markdown format.
4. Documentation Scraping: Fetch the contents of a tutorial or documentation page for offline reading.
5. Content Monitoring: Automatically extract the text from a specific webpage for updates.
6. Data Processing: Parse the HTML of an arbitrary HTTP/HTTPS URL to structure its content.

Website Deep Crawl

Input: one or more start URLs (+ optional scope, depth, page-count, globs, removal selectors). Output: per-page records {url, crawl, metadata, text, markdown, html, outboundLinks} for every page reached within scope.

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

From a small set of start URLs, breadth-first crawl every reachable in-scope page, strip boilerplate (navigation, header, footer, cookie banners, etc.), and emit per-page LLM-ready content (text / markdown / HTML) plus structured metadata — suitable for feeding RAG pipelines, vector databases, or chatbot knowledge bases.

Prerequisites

• One or more start URLs are provided by the caller.
• Target pages are publicly reachable, OR the running browser is already logged in for any pages behind authentication.
• A working directory is available for writing per-page JSON records and the crawl state file.

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification (when prerequisites include login requirement)

If login status for the target site has been confirmed in the current session → skip this step.

Otherwise: open the target site and observe the page login status:

• Logout/sign-out entry, user avatar, or username exists → logged in, continue execution
• Login/register entry exists with no logout entry → not logged in, inform the user that login is needed first, assist the user in completing the login flow

User refuses or cannot log in → terminate execution.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via browser-act --session <name> eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; use the bash tool for execution. The eval token below refers to the browser-act CLI eval subcommand — always include the browser-act --session <name> prefix and the "$(...)" substitution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: discover URLs from /llms.txt

Probes {origin}/llms.txt (a convention used by LLM-friendly documentation sites) and returns every URL it lists. Fast path — try this first.

eval "$(python scripts/discover-llms-txt.py 'https://example.com')"

Parameters:

• positional origin: the site origin (scheme + host), e.g. https://docs.example.com

Output example:

{
  "error": false,
  "source": "llms.txt",
  "count": 88,
  "urls": ["https://docs.example.com/intro", "https://docs.example.com/install"]
}

On failure (file missing or HTTP error): {"error": true, "message": "llms.txt not available (HTTP 404)", "urls": []} — move on to sitemap discovery.

API: discover URLs from /sitemap.xml

Probes {origin}/sitemap.xml and {origin}/sitemap_index.xml, follows nested sitemap indexes, and collects every <loc> URL.

eval "$(python scripts/discover-sitemap.py 'https://example.com' --max-urls 5000)"

Parameters:

• positional origin: site origin
• --max-urls: hard cap to stop ballooning sitemap indexes, default 5000

Output example:

{
  "error": false,
  "source": "sitemap.xml",
  "count": 86,
  "urls": ["https://example.com/page-a", "https://example.com/page-b"]
}

On failure: {"error": true, "message": "No sitemap found at standard paths", "urls": []} — fall back to DOM link discovery.

DOM: discover URLs from the current page

Reads <a href> from the currently loaded page DOM, normalizes them to absolute URLs, drops fragments, asset extensions, and out-of-scope links, and optionally applies include / exclude glob filters. Use when llms.txt and sitemap are both unavailable, or to extend the queue with links discovered while crawling.

eval "$(python scripts/discover-links.py 'https://example.com/docs/' --include-globs '[]' --exclude-globs '["**/changelog/**"]')"

Parameters:

• positional start_url: scope URL. Only links under its directory (or equal to it) are kept.
• --include-globs: JSON array of glob patterns; if non-empty, a link must match at least one to be kept. Default [] (no include filter).
• --exclude-globs: JSON array of glob patterns; matching links are dropped. Default [].

Glob semantics: ** matches any characters (including /), * matches any except /, ? matches one character. Example: https://example.com/{docs,api}/**.

Output example:

{
  "error": false,
  "source": "dom",
  "page": "https://example.com/docs/intro",
  "scope_base": "https://example.com/docs/",
  "count": 12,
  "links": ["https://example.com/docs/intro", "https://example.com/docs/install"]
}

DOM: extract clean content + metadata + outbound links from the current page

The core extractor. Run this on every crawled page after wait stable. Returns the page body in the requested format(s), structured metadata, and the in-scope outbound links found on this page (so callers can extend the BFS queue without a second DOM pass).

eval "$(python scripts/extract-page-content.py 'https://example.com/docs/' --output-format markdown --remove-selectors '.cookie-banner,#chat-widget' --include-globs '[]' --exclude-globs '[]')"

Parameters:

• positional start_url: scope URL — used to filter the outboundLinks array to in-scope links only.
• --output-format: one of markdown, text, html, all. Default markdown. all includes every body field.
• --remove-selectors: comma-separated CSS selectors to delete from the chosen content root before extraction (in addition to the built-in boilerplate list). Use this to strip site-specific chrome (e.g. .cookie-banner, #chat-widget).
• --keep-selector: a single CSS selector identifying the main content area. If set, only this element's content is extracted (overrides the built-in content-root heuristic). Use this when the site has a known main wrapper, e.g. article.docs-content.
• --include-globs / --exclude-globs: same semantics as discover-links; applied to the returned outboundLinks array.

Content-root heuristic (used when --keep-selector is not provided), in priority order: <main>, [role="main"], <article>, #content, .content, <body>.

Built-in boilerplate removal (always applied) includes: nav, header, footer, aside, script, style, noscript, iframe, [role="navigation"], [role="banner"], [role="contentinfo"], .cookie*, .advertisement, .modal, .popup, .share, .social, .breadcrumb, .toc, [aria-hidden="true"], etc.

Output example:

{
  "error": false,
  "url": "https://example.com/docs/intro",
  "crawl": {
    "loadedUrl": "https://example.com/docs/intro",
    "loadedTime": "2026-06-25T04:37:23.643Z",
    "referrerUrl": null
  },
  "metadata": {
    "canonicalUrl": "https://example.com/docs/intro",
    "title": "Introduction — Example Docs",
    "description": "Get started with Example.",
    "author": null,
    "keywords": [],
    "languageCode": "en",
    "publishedAt": null,
    "modifiedAt": null,
    "ogImage": "https://example.com/og.png",
    "ogType": "website"
  },
  "text": "Introduction\n\nGet started with Example…",
  "markdown": "# Introduction\n\nGet started with Example…",
  "outboundLinks": [
    "https://example.com/docs/install",
    "https://example.com/docs/quick-start"
  ]
}

[AI Intervention] On pages with infinite scroll or lazy-loaded sections, before invoking this script: scroll down repeatedly (until page height stops growing or a max-scroll cap is hit) so the dynamic content is in the DOM. The script reads what is currently rendered — it cannot trigger lazy loading on its own.

Composite: full deep crawl from start URL(s)

End-to-end flow. The Agent orchestrates discovery → BFS queue → per-page extraction → persistence. Records are written one per page so that crashes can resume from where they stopped.

Step 1 — seed the queue:

For each start URL, perform discovery in this priority order and merge results. Stop discovery once the queue has enough URLs to honor max_pages.

a. eval "$(python scripts/discover-llms-txt.py '{origin}')" — instant full list when available. b. eval "$(python scripts/discover-sitemap.py '{origin}' --max-urls {cap})" — broad coverage. c. If both fail or return zero in-scope URLs: navigate to the start URL, wait stable, then eval "$(python scripts/discover-links.py '{start_url}' --include-globs '{globs}' --exclude-globs '{globs}')".

Filter all discovered URLs to scope: every URL must start with the start URL's origin + dirname/, and must satisfy include / exclude globs.

Step 2 — initialize state:

Create the following in the working directory:

• crawl_state.json — {visited: [], queue: [...seedUrls], output_dir: "...", config: {...}}
• pages/ directory — one JSON file per successfully crawled page, named by URL hash

Step 3 — BFS loop (one URL at a time, in queue order, until max_pages reached or queue empty):

For each url popped from the queue:

a. Skip if url is in visited or its metadata.canonicalUrl (from a prior page) is already in visited. b. navigate {url} → wait stable (use --timeout 60000 for slow sites). c. (Optional, only when the target has lazy-loaded content) scroll down until height stable or 10 scrolls done. d. eval "$(python scripts/extract-page-content.py '{start_url}' --output-format {format} --remove-selectors '{selectors}' --include-globs '{globs}' --exclude-globs '{globs}')". e. If result error: true → record the failure into crawl_state.json#failed and continue. Do NOT retry blindly. f. Write the JSON record to pages/{hash}.json. g. Append url and metadata.canonicalUrl to visited. h. For each link in outboundLinks: if not in visited and not already in queue, append to queue. Cap queue size at max_pages * 4 to bound memory. i. Persist crawl_state.json after every page (resume on next run if interrupted).

Step 4 — finalize:

Emit a summary {total_pages, success_count, failed_count, duration_seconds, output_dir}. Optionally concatenate all pages/*.json into a single dataset.jsonl for downstream loading.

Configuration parameters (set by the Agent before Step 1 based on user request):

• start_urls: list of seed URLs (one or more)
• max_pages: hard cap on pages crawled, default 100
• max_depth: hard cap on link depth from start URL, default unlimited (-1)
• include_globs: JSON array, default []
• exclude_globs: JSON array, default []
• output_format: markdown | text | html | all, default markdown
• remove_selectors: comma-separated site-specific selectors to strip, default ""
• keep_selector: optional content-root selector, default ""
• output_dir: where to write pages/ and crawl_state.json, default ./output/{site}-crawl/

Output example (per-page record, written to pages/{hash}.json):

{
  "url": "https://example.com/docs/intro",
  "crawl": { "loadedUrl": "...", "loadedTime": "...", "referrerUrl": null, "depth": 0 },
  "metadata": { "title": "...", "description": "...", "languageCode": "en", "canonicalUrl": "..." },
  "text": "...",
  "markdown": "# ...",
  "outboundLinks": ["..."]
}

Summary example:

{
  "total_pages": 86,
  "success_count": 84,
  "failed_count": 2,
  "duration_seconds": 412,
  "output_dir": "./output/example-crawl/"
}

Pagination

This is a recursive crawler, not a list with pages. Boundary control is by max_pages (queue length cap) and max_depth (links-away-from-start cap), not by API pagination. Termination: queue empty OR max_pages reached OR no more in-scope outbound links discovered.

Success Criteria

success_count >= 1 AND success_count / total_pages >= 0.8 AND extracted markdown body length per page > 100 chars for at least 80% of pages

Known Limitations

• Pages behind authentication require the running browser to be logged in beforehand — this Skill does not handle login flows.
• Pages whose content is rendered after async user interaction beyond simple scroll (e.g. clicking "Load more", expanding accordions to reveal content) need the Agent to add the relevant click before invoking extract-page-content; otherwise the hidden content will be missing.
• <iframe> content is removed by default (treated as boilerplate). If a page's main content lives inside an iframe, the Agent must first navigate into the iframe URL and crawl it separately.
• File downloads (PDF, DOCX, XLSX) are not handled. URLs with these extensions are intentionally filtered from the crawl queue.
• Some sites' boilerplate is structurally indistinguishable from main content (e.g. "Was this page helpful?" footers placed inside <main>). The Agent should pass site-specific patterns via --remove-selectors to strip them.
• Single-page applications that load content via JS after navigation may need a longer wait stable --timeout. Anti-scraping CAPTCHAs are not bypassed by this Skill; the calling browser must already pass them.

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over
• Prefer llms.txt / sitemap.xml over DOM discovery: A single fetch returns the full URL list; DOM discovery requires loading every page first. Always try the two API discovery routes first and only fall back to DOM when both return empty.
• Polite delay between pages: Default to 500–1500 ms between page navigations to avoid burst patterns. Tighten only on sites you control or own.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/webcrawler-deep-crawl.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Meta Ad Library — Search Ads

keyword or page ID → ad list with creatives, metrics, and pagination cursor

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Fetch ads from Meta Ad Library for a keyword search or a specific Facebook page, returning structured ad data with all available fields.

Prerequisites

• Browser is open on any Facebook page: https://www.facebook.com/
• No login required for this capability

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; it is recommended to use the bash tool for execution.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read scripts/*.py source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

API: search ads by keyword

eval "$(python scripts/search-ads.py --query '{keyword}' --country {country} --first {count})"

Parameters:

• --query: keyword to search (mutually exclusive with --page-id)
• --country: 2-letter ISO country code or ALL, default ALL
• --active-status: active | inactive | all, default active
• --ad-type: ALL | POLITICAL_AND_ISSUE_ADS | HOUSING_ADS, default ALL
• --media-type: all | image | video | meme, default all
• --platforms: space-separated list, e.g. facebook instagram, default all platforms
• --cursor: pagination cursor from previous response's end_cursor, default first page
• --first: number of ads to return per page, default 10

Output example:

{
  "error": false,
  "count": 10,
  "has_next_page": true,
  "end_cursor": "AQHSmvKYSBolzAS9Wq8VSnt4...",
  "ads": [
    {
      "ad_archive_id": "1869276447125570",  // unique ad archive ID
      "ad_id": null,                         // ad ID (null for some ads)
      "page_id": "15087023444",              // advertiser's Facebook page ID
      "page_name": "Nike",                   // advertiser's page name
      "page_profile_uri": "https://facebook.com/nike",
      "page_profile_picture_url": "https://scontent.xx.fbcdn.net/...",
      "is_active": true,                     // whether ad is currently running
      "start_date": 1773730800,              // ad start date (unix timestamp)
      "end_date": 1779692400,                // ad end date (unix timestamp), null if still active
      "publisher_platform": ["facebook", "instagram"],  // platforms the ad runs on
      "currency": "USD",                     // spend currency, null if not disclosed
      "spend": null,                         // estimated spend range, null if not disclosed
      "impressions_with_index": null,        // impression estimate, null if not disclosed
      "reach_estimate": null,                // reach estimate, null if not disclosed
      "categories": [],                      // ad categories
      "contains_sensitive_content": false,
      "body": "Get the gear that never misses.",  // main ad body text
      "caption": "nike.com",                // ad caption
      "title": "Nike Air Monarch IV",        // ad title
      "cta_text": "Shop Now",               // CTA button text
      "cta_type": "SHOP_NOW",               // CTA button type
      "link_url": "https://www.nike.com/...",// destination URL
      "display_format": "carousel",          // ad format
      "cards": [...],                        // carousel cards (each has body, title, cta_type, link_url, original_image_url, video_hd_url)
      "images": [...],                       // image creatives
      "videos": [...]                        // video creatives
    }
  ]
}

API: search ads by page ID

eval "$(python scripts/search-ads.py --page-id '{page_id}' --country {country} --first {count})"

Parameters:

• --page-id: Facebook page ID (mutually exclusive with --query). To find a page ID: navigate to the Facebook page, the ID appears in the URL or in the page's "About" section
• All other parameters same as keyword search above

Output example: same structure as keyword search above.

Enum Parameters

--active-status: active | inactive | all

--ad-type: ALL | POLITICAL_AND_ISSUE_ADS | HOUSING_ADS

--media-type: all | image | video | meme

--platforms: facebook | instagram | whatsapp | messenger | audience_network | threads (pass multiple values space-separated)

--country: ISO 2-letter country code (e.g. US, GB, DE, ALL) [collection partially done — full list of supported country codes follows ISO 3166-1 alpha-2 standard]

Pagination

API Pagination: --cursor, type: cursor, start value: omit for first page. Next page value source: end_cursor field in response. Termination: has_next_page: false.

Success Criteria

error = false AND count >= 0 AND has_next_page field present

(count may be 0 for valid queries that return no results; this is not an error)

Known Limitations

• spend, impressions_with_index, reach_estimate fields are frequently null — Meta only discloses these for political/social issue ads or at their discretion
• ad_id is often null — ad_archive_id is the reliable unique identifier
• Rate limiting may occur with very high-frequency requests; add 1-2 second delays between requests in batch scripts
• Private or restricted ads may not appear in results
• The doc_id (27201872659451053) is a compiled query ID that may change when Meta updates their frontend; if errors occur, re-explore to find the updated ID

Execution Efficiency

• Batch orchestration: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
• Test before batch execution: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
• Reduce redundant pre-operations: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
• Error resumption: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/facebook-ads-scraper-facebook-ads-library-search.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line: {YYYY-MM-DD}: {what happened} -> {conclusion}

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.

Facebook Groups — Scrape Posts

Input: Facebook group URL + sort order + desired count → Output: post list with full metadata (JSON).

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Given a Facebook group URL, scrape N posts sorted by the specified order and return structured metadata for each post.

Prerequisites

• Target group page is already open in the browser: https://www.facebook.com/groups/{group_slug_or_id}
• Already logged into Facebook (user avatar, Messenger icon, and notification bell visible in the top-right corner)

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If Facebook login status has been confirmed in the current session → skip this step.

Otherwise, navigate to https://www.facebook.com/ and verify login status programmatically:

browser-act navigate 'https://www.facebook.com/'
browser-act wait stable --timeout 15000
browser-act eval "JSON.stringify({user_id: document.cookie.match(/c_user=(\d+)/)?.[1] || '0', USER_ID: (()=>{try{return require('CurrentUserInitialData').USER_ID;}catch(e){return '0';}})()})"

Verdict:

• user_id is a non-empty numeric string (e.g., "61560817072276"), or USER_ID !== "0" → logged in, continue
• user_id === null or USER_ID === "0" → not logged in; assist user: run browser-act browser open {browser_id} https://www.facebook.com/login --headed to open a headed window so the user can sign in manually (Stealth normal mode persists cookies — one login is reusable)

Facebook may clear c_user mid-session: if GraphQL errors such as field_exception or missing_required_variable_value occur during execution, re-run this login check before assuming the script is broken.

User refuses or cannot log in → terminate execution. Facebook enforces strict restrictions on unauthenticated group access (login modal blocks pagination, feed returns partial data + field_exception); login is a hard prerequisite.

Capability Components

This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the authenticated user, never bypassing authentication or access controls — equivalent to copy-pasting on the user's behalf. JS code is encapsulated in Python files under the scripts/ directory, invoked via eval "$(python scripts/xxx.py {params})". $(...) is bash syntax; use the bash tool for execution.

API: Scrape group posts (with auto-pagination)

Navigate to the target group page first, then invoke the scrape script (it auto-resolves the numeric group ID from the current page):

browser-act navigate 'https://www.facebook.com/groups/{group_slug_or_id}'
browser-act wait stable --timeout 20000
browser-act eval "$(python scripts/scrape-posts.py --sort CHRONOLOGICAL --count 20)"

Parameters:

• --sort: Sort order, default CHRONOLOGICAL. See "Enum Parameters" below
• --count: Desired number of posts, default 20. Script auto-paginates until count is met or feed is exhausted
• --max-pages: Pagination safety cap, default 100
• --doc-id: GraphQL persisted query doc_id for GroupsCometFeedRegularStoriesPaginationQuery, default 26577462205242925. Update via this flag if Facebook rotates the version (see "Known Limitations")