Python SDK
pip install webscans โ typed, sync + async, MIT-licensed. Every one of the 12 REST endpoints wrapped with clean Python ergonomics and Pydantic response models.
# SDK + CLI
pip install webscans
# + MCP server (adds mcp>=1.2 dependency)
pip install "webscans[mcp]"
๐ฆ PyPI ยท ๐ป GitHub (MIT) ยท Python 3.10+
Authentication
from webscans import WebScans
ws = WebScans(api_key="ws_...") # explicit
ws = WebScans() # picks up $WEBSCANS_API_KEY
Public endpoints (search, domain, similar, enrich) work without a key at reduced limits. Paid endpoints (large exports, my-companies) require one.
Quickstart
from webscans import WebScans
with WebScans(api_key="ws_...") as ws:
# Real-time enrichment cascade
p = ws.enrich("stripe.com")
print(p.source, p.elapsed_ms, p.profile["industry"])
# cached 8 Finance & Banking
# Keyword + filter search
r = ws.search("solar panels", country="US", tech="HubSpot", limit=25)
for hit in r.results:
print(hit.domain, hit.title)
# Lookalikes via FAISS embedding similarity
for c in ws.similar("figma.com", k=10).competitors:
print(f"{c.domain:<28} {c.score:.3f} {c.match_type}")
# Natural-language (Claude picks the tool)
r = ws.copilot("find SaaS companies in California using HubSpot")
print(r.answer)
search
r = ws.search(
"carbon accounting", # required โ keyword or phrase
mode="phrase", # phrase | any | single
tech="Segment", # substring match on tech_stack
industry="Software & Information Technology",
country="US", # ISO-2
tld="io", # no dot
has="pricing", # email | phone | pricing | blog | ecommerce | contact
limit=100,
offset=0,
)
print(r.result_count, r.total_count, r.search_time_ms)
for hit in r.results:
print(hit.domain, hit.industry)
Returns a SearchResponse with results: list[SearchResult]. Every field is a typed Pydantic attribute โ full autocomplete in your IDE.
count
ws.count("hubspot") # โ 19433
domain
p = ws.domain("stripe.com")
print(p.title, p.industry, p.tech_stack, p.emails)
Cache-only. Raises WebScansNotFoundError if the domain isn't indexed โ use enrich() to fetch fresh.
similar
r = ws.similar("figma.com", k=25)
print(r.method, r.total_found) # faiss_embedding, 25
for c in r.competitors:
print(c.domain, c.score, c.match_type)
Uses FAISS IVF_PQ over 13.2M float16 gte-Qwen2-7B-instruct embeddings. Scored 0โ1 cosine. Match types: competitor (โฅ0.82), lookalike (โฅ0.70), similar (โฅ0.55).
lookup
p = ws.lookup("acme.io") # cache first, live fallback
p = ws.lookup("acme.io", force=True) # bypass cache
enrich
r = ws.enrich("acme.io", wait_ms=800)
print(r.source, r.elapsed_ms, r.enriched_columns)
# live_curl 102 37
print(r.profile["title"], r.profile["industry"])
wait_ms (default 800): live curl_cffi + Common Crawl WARC. First byte wins โ
3. Playwright browser fallback (JS-heavy / bot-walled) โ
4. Bright Data residential proxy last resort.
autocomplete
for h in ws.autocomplete("stri", limit=6):
print(h.domain, h.title)
copilot
r = ws.copilot("show me healthcare companies with pricing pages")
print(r.answer) # natural-language summary
print(r.tool_call) # which tool Claude picked + args
print(r.results) # structured tool output
export_search
# write directly to disk
ws.export_search("solar installers", tier="pro", out="solar.csv")
# or grab bytes in memory
blob = ws.export_search("solar installers", tier="pro")
bulk_search
ws.bulk_search(
["hvac", "roofing", "solar"],
tier="pro",
out="home-services.csv",
)
bulk_lookup
# upload up to 100K domains, get back enriched CSV
ws.bulk_lookup("my_domains.csv", out="enriched.csv")
export_my_companies
ws.export_my_companies(out="saved.csv") # everything
ws.export_my_companies(search="acme", out="acme.csv") # substring filter
ws.export_my_companies(query="fintech", out="ft.csv") # by original saved search
Paid tier โ 1 credit per row. Response headers surface X-Credits-Needed / X-Row-Count if you're short.
AsyncWebScans
import asyncio
from webscans import AsyncWebScans
async def main():
async with AsyncWebScans(api_key="ws_...") as ws:
# Fan out 50 enrichments concurrently
results = await asyncio.gather(*[
ws.enrich(d) for d in ["stripe.com","shopify.com","figma.com",...]
])
for r in results:
print(r.domain, r.source, r.elapsed_ms)
asyncio.run(main())
Identical surface to WebScans, powered by httpx.AsyncClient. Prefer async when fanning out โฅ5 concurrent requests.
Exception hierarchy
from webscans import (
WebScansError, # base
WebScansAuthError, # 401 / 403
WebScansCreditError, # 402
WebScansNotFoundError, # 404
WebScansRateLimitError, # 429 โ has .retry_after_seconds
WebScansServerError, # 5xx
)
try:
p = ws.domain("someweird.io")
except WebScansRateLimitError as e:
time.sleep(e.retry_after_seconds)
except WebScansNotFoundError:
p = ws.enrich("someweird.io") # fall through to live fetch