OpenPhish_v2

OpenPhish uses proprietary Artificial Intelligence algorithms to automatically identify zero-day phishing sites and provide comprehensive, actionable, real-time threat intelligence.

Data Enrichment & Threat Intelligence · OpenPhish

Details

ID	`OpenPhish_v2`
Provider	OpenPhish
Category	Data Enrichment & Threat Intelligence
From Version	5.0.0
Docker Image	`demisto/python3:3.12.13.10116658`
Supported Modules	Agentix XSIAM

README

OpenPhish uses proprietary Artificial Intelligence algorithms to automatically identify zero-day phishing sites and provide comprehensive, actionable, real-time threat intelligence.

Configure OpenPhish_v2 in Cortex

Parameter	Description	Required
https	Use HTTPS connection	False
fetchIntervalHours	Database refresh interval (hours)	False
proxy	Use system proxy settings	False
insecure	Trust any certificate (not secure)	False

Commands

You can execute these commands from the CLI, as part of an automation, or in a playbook.
After you successfully execute a command, a DBot message appears in the War Room with the command details.

url

Checks the reputation of a URL.

Notice: Submitting indicators using this command might make the indicator data publicly available. See the vendor’s documentation for more details.

Base Command

url

Input

Argument Name	Description	Required
url	URL to check.	Required

Context Output

Path	Type	Description
URL.Data	unknown	The URL
URL.Malicious.Vendor	unknown	The vendor reporting the URL as malicious.
URL.Malicious.Description	unknown	A description of the malicious URL.
DBotScore.Indicator	unknown	The indicator that was tested.
DBotScore.Type	unknown	The indicator type.
DBotScore.Vendor	unknown	The vendor used to calculate the score.
DBotScore.Score	unknown	The actual score.

Command Example

!url using-brand=OpenPhish_v2 url="google.com, hxxp://hang3clip.ddns.net/"

Context Example

{
    "DBotScore": [
        {
            "Indicator": "google.com",
            "Score": 0,
            "Type": "url",
            "Vendor": "OpenPhish"
        },
        {
            "Indicator": "hxxp://hang3clip.ddns.net/",
            "Score": 3,
            "Type": "url",
            "Vendor": "OpenPhish"
        }
    ],
    "URL": [
        {
            "Data": "google.com"
        },
        {
            "Data": "hxxp://hang3clip.ddns.net/",
            "Malicious": {
                "Description": "Match found in OpenPhish database",
                "Vendor": "OpenPhish"
            }
        }
    ]
}

Human Readable Output

OpenPhish Database - URL Query

No matches for URL google.com

Found matches for given URL hxxp://hang3clip.ddns.net/

openphish-reload

Reload OpenPhish database

Base Command

openphish-reload

Input

| Argument Name | Description | Required |
| — | — | — |

Command Example

!openphish-reload

Human Readable Output

updated successfully

openphish-status

Show OpenPhish database status

Base Command

openphish-status

Input

| Argument Name | Description | Required |
| — | — | — |

Context Output

There is no context output for this command.

Command Example

!openphish-status

Human Readable Output

Configuration parameters

https — Use HTTPS connection
fetchIntervalHours — Database refresh interval (hours)
proxy — Use system proxy settings
insecure — Trust any certificate (not secure)
integrationReliability — Source Reliability
feedExpirationPolicy —
feedExpirationInterval —

Commands (3)

openphish-reload
Reload OpenPhish database.
openphish-status
Show OpenPhish database status.
url
Checks the reputation of a URL.

Source

OpenPhish_v2.py OpenPhish_v2_test.py OpenPhish_v2.yml OpenPhish_v2_description.md command_examples.txt

Download History Raw

from datetime import datetime

import demistomock as demisto
import OpenPhish_v2
import pytest
from freezegun import freeze_time
from OpenPhish_v2 import (
    Client,
    _is_reload_needed,
    reload_command,
    remove_backslash,
    status_command,
    url_command,
)
from test_data.api_raw import RAW_DATA

MOCK_URL = "http://openphish.com"
MOCK_DELIVERED_MESSAGE = {}
DBOT_KEY = "DBotScore(val.Indicator && val.Indicator == obj.Indicator && val.Vendor == obj.Vendor && val.Type == obj.Type)"

RELOADED_DATA = [
    (Client(MOCK_URL, True, False, 2), {}, True),  # case no data in memory
    (
        Client(MOCK_URL, True, False, 2),
        {"list": []},
        True,
    ),  # case no timestamp and list is emtpy
    (
        Client(MOCK_URL, True, False, 2),
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": None,
        },
        True,
    ),  # case no timestamp
    (
        Client(MOCK_URL, True, False, 1),
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": 1601542800000,
        },  # datetime(2020, 10, 1, 10, 00, 00, 0) - timedelta(hours=1)
        True,
    ),
    (
        Client(MOCK_URL, True, False, 2),
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": 1601542800000,
        },  # datetime(2020, 10, 1, 10, 00, 00, 0) - timedelta(hours=1)
        False,
    ),
    (
        Client(MOCK_URL, True, False, 0.5),
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": 1601542800000,
        },  # datetime(2020, 10, 1, 10, 00, 00, 0) - timedelta(hours=1)
        True,
    ),
]
INTEGRATION_NAME = "OpenPhish"


@pytest.fixture(autouse=True)
def handle_calling_context(mocker):
    mocker.patch.object(demisto, "callingContext", {"context": {"IntegrationBrand": INTEGRATION_NAME}})


@pytest.mark.parametrize("client,data,output", RELOADED_DATA)
def test_is_reload_needed(mocker, client, data, output):
    """
    Given:
        - data as IntegrationContext

    When:
        - reload command was required

    Then:
        - Returns False if last reload occurred in the past fetch_interval_hours. True otherwise

    """
    with freeze_time(datetime(2020, 10, 1, 10, 00, 00, 0)):
        assert _is_reload_needed(client, data) == output


LINKS = [("goo.co/", "goo.co"), ("goo.co", "goo.co")]


@pytest.mark.parametrize("url, expected_result", LINKS)
def test_remove_backslash(url: str, expected_result: str):
    """
    Given:
        - string representing url

    When:
        - saving data from to the integration context or checking a specific url

    Then:
        - checks the url format is without a backslash as last character

    """
    assert remove_backslash(url) == expected_result


def test_reload_command(mocker):
    """
    When:
        - reloading data from to the api to integration context

    Then:
        - checks if the reloading finished successfully

    """
    mock_data_from_api = RAW_DATA
    mocker.patch.object(Client, "http_request", return_value=mock_data_from_api)
    mocker.patch.object(demisto, "setIntegrationContext")
    client = Client(url=MOCK_URL, use_ssl=False, use_proxy=False, fetch_interval_hours=1)
    status = reload_command(client)
    assert status.readable_output == "Database was updated successfully to the integration context."


STANDARD_NOT_LOADED_MSG = "OpenPhish Database Status\nDatabase not loaded.\n"
STANDARD_4_LOADED_MSG = "OpenPhish Database Status\nTotal **4** URLs loaded.\nLast load time **Thu Oct 01 2020 06:00:00 (UTC)**\n"
CONTEXT_MOCK_WITH_STATUS = [
    ({}, STANDARD_NOT_LOADED_MSG),  # case no data in memory
    (
        {"list": [], "timestamp": "1601532000000"},
        STANDARD_NOT_LOADED_MSG,
    ),  # case no timestamp and list is emtpy
    (
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": "1601532000000",
        },  # datetime(2020, 10, 1, 10, 00, 00, 0) - timedelta(hours=1)}
        STANDARD_4_LOADED_MSG,
    ),
]


@pytest.mark.parametrize("data,expected_result", CONTEXT_MOCK_WITH_STATUS)
@freeze_time("1993-06-17 11:00:00 GMT")
def test_status_command(mocker, data, expected_result):
    """
    Given:
        - Integration context
    When:
        - After status command
    Then:
        - Returns number of loaded urls if data was loaded.
        - Otherwise, returns Database not loaded.
    """
    client = Client(MOCK_URL, True, False, 1)
    mocker.patch.object(demisto, "getIntegrationContext", return_value=data)
    status = status_command(client)
    assert status.readable_output == expected_result


CONTEXT_MOCK_WITH_URL = [
    (
        {"url": "hxxp://lloyds.settlemypayee.uk"},
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": "1601532000000",
        },
        [
            {
                "URL": [
                    {
                        "Data": "hxxp://lloyds.settlemypayee.uk",
                        "Malicious": {
                            "Vendor": "OpenPhish",
                            "Description": "Match found in OpenPhish database",
                        },
                    }
                ],
                "DBOTSCORE": [
                    {
                        "Indicator": "hxxp://lloyds.settlemypayee.uk",
                        "Type": "url",
                        "Vendor": "OpenPhish",
                        "Score": 3,
                    }
                ],
            }
        ],
    ),
    (
        {"url": "hxxp://goo.co"},
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": "1601532000000",
        },
        [
            {
                "URL": [{"Data": "hxxp://goo.co"}],
                "DBOTSCORE": [
                    {
                        "Indicator": "hxxp://goo.co",
                        "Type": "url",
                        "Vendor": "OpenPhish",
                        "Score": 0,
                    }
                ],
            }
        ],
    ),
    (
        {"url": "hxxp://whatsapp-chat02.zzux.com,hxxp://lloyds.settlemypayee.uk"},
        {
            "list": [
                "hxxp://www.niccakorea.com/board/index.html",
                "hxxp://lloyds.settlemypayee.uk",
                "hxxp://whatsapp-chat02.zzux.com",
                "hxxp://dd0ddddddcuser.ey.r.appspot.com",
            ],
            "timestamp": "1601532000000",
        },
        [
            {
                "URL": [
                    {
                        "Data": "hxxp://whatsapp-chat02.zzux.com",
                        "Malicious": {
                            "Vendor": "OpenPhish",
                            "Description": "Match found in OpenPhish database",
                        },
                    }
                ],
                "DBOTSCORE": [
                    {
                        "Indicator": "hxxp://whatsapp-chat02.zzux.com",
                        "Score": 3,
                        "Type": "url",
                        "Vendor": "OpenPhish",
                    }
                ],
            },
            {
                "URL": [
                    {
                        "Data": "hxxp://lloyds.settlemypayee.uk",
                        "Malicious": {
                            "Vendor": "OpenPhish",
                            "Description": "Match found in OpenPhish database",
                        },
                    }
                ],
                "DBOTSCORE": [
                    {
                        "Indicator": "hxxp://lloyds.settlemypayee.uk",
                        "Score": 3,
                        "Type": "url",
                        "Vendor": "OpenPhish",
                    }
                ],
            },
        ],
    ),
]


@pytest.mark.parametrize("url,context,expected_results", CONTEXT_MOCK_WITH_URL)
def test_url_command(mocker, url, context, expected_results):
    """
    Given:
        - a url

    When:
        - mocking the integration context data, runnig url_command

    Then:
        - validating whether the url is malicious (in integration context)

    """
    mocker.patch.object(
        demisto,
        "getIntegrationContext",
        return_value=context,
    )
    mocker.patch.object(OpenPhish_v2, "_is_reload_needed", return_value=False)
    client = Client(MOCK_URL, True, False, 1)
    results = url_command(client, **url)
    assert len(results) >= 1
    for i in range(len(results)):
        output = results[i].to_context().get("EntryContext", {})
        assert output.get("URL(val.Data && val.Data == obj.Data)", []) == expected_results[i].get("URL")
        assert output.get(DBOT_KEY, []) == expected_results[i].get("DBOTSCORE")