Tavily

Tavily is a web service that provides real-time web search and retrieval capabilities through an API, enabling developers to fetch and extract relevant information from the internet in structured formats like JSON.

Utilities · Tavily

Details

ID	`Tavily`
Provider	Tavily
Category	Utilities
From Version	6.1.0
Docker Image	`demisto/python3:3.12.13.10116658`
Supported Modules	Agentix Cloud Runtime Security Cloud Posture Security XSIAM EDR Cortex Cloud

README

Tavily is a web service that provides real-time web search and retrieval capabilities through an API, enabling
developers to fetch and extract relevant information from the internet.

Configure Tavily in Cortex

Parameter	Description	Required
Server URL		True
API Key	The API Key to use for the connection	True
Trust any certificate (not secure)		False
Use system proxy settings		False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

tavily-extract

Extract web page content from a specified URL.

Base Command

tavily-extract

Input

Argument Name	Description	Required
url	The url to extract its content.	Required

Context Output

Path	Type	Description
Tavily.URL	string	The URL from which the content was extracted.
Tavily.Content	string	The full content extracted from the URL.

Configuration parameters

url — Server URL (required)
api_key — API Key (required)
insecure — Trust any certificate (not secure)
proxy — Use system proxy settings

Commands (1)

tavily-extract
Extract web page content from a specified URL.

Source

Tavily.py Tavily_test.py Tavily.yml Tavily_description.md command_examples.txt

Download History Raw

import json
from requests.models import Response
import pytest
from Tavily import TavilyExtractClient, extarct_command


@pytest.fixture
def tavily_extract_client():
    return TavilyExtractClient(api_key="test-api-key")


def test_extract_success(mocker, tavily_extract_client):
    """
    Sanity test for success extract
    """
    response = Response()
    response.status_code = 200
    response._content = json.dumps(
        {"results": [{"url": "https://example.com", "raw_content": "Example content from the web page."}]}
    ).encode("utf-8")

    http_request = mocker.patch.object(tavily_extract_client, "_http_request", return_value=response)

    url = "https://example.com"
    result = tavily_extract_client.extract(url)

    assert "results" in result
    assert result["results"][0]["url"] == "https://example.com"
    assert "raw_content" in result["results"][0]
    http_request.assert_called_once()


def test_extract_failure(mocker, tavily_extract_client):
    """
    Sanity test for failure extract
    """
    response = Response()
    response.status_code = 400
    response._content = b"Bad Request"

    http_request = mocker.patch.object(tavily_extract_client, "_http_request", return_value=response)

    url = "https://bad-url.com"

    with pytest.raises(Exception) as excinfo:
        tavily_extract_client.extract(url)

    assert "Request failed" in str(excinfo.value)
    http_request.assert_called_once()


def test_extarct_command(mocker, tavily_extract_client):
    """
    Given:
        - A client
        - An url for check
    When:
        - Executing the extarct_command function
    Then:
        - Ensure the command results contains two outputs (URL and Content)
        - Ensure the first dictionary for the first given url contains the two keys: URL and Content and correct values
    """
    response = Response()
    response.status_code = 200
    response._content = json.dumps(
        {"results": [{"url": "https://example.com", "raw_content": "Example content from the web page."}]}
    ).encode("utf-8")

    mocker.patch.object(tavily_extract_client, "_http_request", return_value=response)

    command_results = extarct_command(client=tavily_extract_client, args={"url": "https://example.com"})

    assert len(command_results.outputs) == 2
    assert isinstance(command_results.outputs, dict)
    assert len(command_results.outputs.keys()) == 2
    assert command_results.outputs["URL"] == "https://example.com"
    assert command_results.outputs["Content"] == "Example content from the web page."