SpiderGo Docs

Backend API

Base URL

EnvironmentBase URL
Localhttp://localhost:8080
ProductionYour deployed API domain

API Conventions

ItemConvention
Content Typeapplication/json
Cookie AuthAccess/refresh via HttpOnly cookies
API Key AuthAuthorization: Bearer sk_live_...
Error Body{ "message": "..." }

Authentication

Login sets access and refresh cookies. Protected dashboard endpoints enforce cookie validation via auth middleware.

API Key Auth (Programmatic)

Used by /v1/* endpoints.

Authorization: Bearer sk_live_your_key

Endpoint Reference

Auth and Session Endpoints

MethodPathAuthPurpose
POST/auth/registerPublicCreate account
POST/auth/loginPublicLogin and set auth cookies
GET/auth/meCookieGet current profile
GET/auth/refreshCookieRefresh/rotate session tokens
GET/auth/oauthPublicOAuth provider redirect
GET/auth/oauth/googlePublicGoogle OAuth callback flow
GET/auth/oauth/githubPublicGitHub OAuth callback flow
POST/auth/resend-emailCookie/Public flowResend verification code
POST/auth/verify-emailPublicVerify email with code
POST/auth/forgot-passwordPublicStart password reset
POST/auth/verify-reset-codePublicVerify reset code
POST/auth/reset-passwordPublicComplete password reset
MethodPathAuthPurpose
POST/scrapeCookieRun single-page scrape job
POST/crawlCookieRun crawl job
GET/historyCookieGet current user history

Trial Endpoints (Public, Rate-Limited)

MethodPathAuthPurpose
POST/trial/scrapePublicTrial single-page scrape
POST/trial/crawlPublicTrial crawl
MethodPathAuthPurpose
POST/auth/api-keysCookieCreate API key
GET/auth/api-keysCookieList API keys (metadata only)
DELETE/auth/api-keys/:idCookieRevoke API key

Versioned API (API Key Auth)

MethodPathAuthPurpose
POST/v1/scrapeAPI KeyProgrammatic scrape
POST/v1/crawlAPI KeyProgrammatic crawl
GET/v1/historyAPI KeyProgrammatic history retrieval

Common Request and Response Examples

Register: POST /auth/register

Request:

{
  "name": "Jane Doe",
  "email": "jane@example.com",
  "password": "StrongPassword123!"
}

Response:

{
  "message": "User registered successfully"
}

Login: POST /auth/login

Request:

{
  "email": "jane@example.com",
  "password": "StrongPassword123!"
}

Response body includes user metadata and sets HttpOnly cookies:

{
  "user": {
    "user_id": "...",
    "first_name": "Jane",
    "last_name": "Doe",
    "email": "jane@example.com",
    "is_verified": true,
    "avatar_url": ""
  }
}

Profile: GET /auth/me

Requires access token cookie.

Returns current user profile:

{
  "user_id": "...",
  "first_name": "Jane",
  "last_name": "Doe",
  "email": "jane@example.com",
  "is_verified": true,
  "avatar_url": ""
}

Email Verify Payload: POST /auth/verify-email

{
  "email": "jane@example.com",
  "code": 123456
}

Scrape/Crawl Payload

Request:

{
  "url": "https://example.com"
}

Both /scrape and /crawl return a job envelope:

{
  "message": {
    "CRID": "...",
    "UserID": "...",
    "Pages": []
  }
}

Create API Key: POST /auth/api-keys

Request:

{
  "name": "Production Integrations"
}

Response includes raw key once:

{
  "message": "API key created. Store it securely now; it will not be shown again.",
  "api_key": "sk_live_...",
  "meta": {
    "key_id": "...",
    "name": "Production Integrations",
    "prefix": "sk_live_",
    "last4": "abcd",
    "daily_limit": 1000,
    "is_active": true,
    "created_at": "2026-03-23T00:00:00Z"
  }
}

Error Handling

Common error payload:

{
  "message": "Error details"
}
StatusMeaning
400Invalid input
401Unauthorized
409Conflict (for example, duplicate registration)
429Rate limit or trial quota exceeded
500Server-side failure