WordTokenizer Deprecated

Deprecated. Use DBotPreProcessTextData instead.

Details

ID	`WordTokenizer`
Language	python
From Version	5.0.0
Docker Image	`demisto/nltk:2.0.0.86396`
Tags	phishing ml

Source

WordTokenizeTest.py WordTokenizeTest_test.py WordTokenizeTest.yml

Download History Raw

commonfields:
  id: WordTokenizer
  version: -1
name: WordTokenizer
script: ''
type: python
subtype: python3
tags:
- phishing
- ml
comment: Deprecated. Use DBotPreProcessTextData instead.
enabled: true
args:
- name: value
  required: true
  description: The text value.
  isArray: true
- name: type
  auto: PREDEFINED
  predefined:
  - word
  - punkt
  description: 'Tokenizer type. you can read more about this here: https://www.nltk.org/api/nltk.tokenize.html'
  defaultValue: word
- name: cleanHtml
  auto: PREDEFINED
  predefined:
  - "yes"
  - "no"
  description: Clean html from text value?
  defaultValue: "yes"
- name: removeLineBreaks
  auto: PREDEFINED
  predefined:
  - "yes"
  - "no"
  description: Remove line breaks?
  defaultValue: "yes"
- name: encoding
  description: Text encoding.
  defaultValue: utf-8
- name: hashWordWithSeed
  description: If non-empty hash the words with this seed.
outputs:
- contextPath: WordTokenizeOutput
  description: 'Output text.'
  type: string
scripttarget: 0
runonce: true
dockerimage: demisto/nltk:2.0.0.86396
fromversion: 5.0.0
tests:
- No tests (deprecated)
deprecated: true