Tools

​

Overview

​

Available tools

​

Data extraction

​

Data transformation

​

Data combination

​

Data transformation

​

HTML Cleaner

• Scripts and styles
• Comments
• Empty tags
• Specific selectors

​

HTML to Markdown

Input: <h1>Title</h1><p>Paragraph</p>
Output: # Title\n\nParagraph

​

Find and Replace

​

JSON Path Extractor

// Input
{
  "results": [
    {"name": "Alice", "score": 95},
    {"name": "Bob", "score": 87}
  ]
}

// Path: $.results[0].name
// Output: "Alice"

// Path: $.results[*].score
// Output: [95, 87]

​

Common patterns

​

API data pipeline

​

Best practices

​

Web scraping

• Respect robots.txt and rate limits
• Add delays between requests (2-3 seconds)
• Handle errors gracefully (404, timeout)
• Cache results when possible

​

API calls

• Use authentication properly
• Handle rate limits with retries
• Validate responses before processing
• Log failures for debugging

​

Next steps