new Stats(message_obj)
Used for cluster stats for the web app.
Parameters:
| Name | Type | Description |
|---|---|---|
message_obj |
Message |
- Source:
Methods
activeBots(fn)
Returns active bots info.
Parameters:
| Name | Type | Description |
|---|---|---|
fn |
function | callback |
- Source:
clusterInfo(fn)
Returns cluster info.
Parameters:
| Name | Type | Description |
|---|---|---|
fn |
function | callback |
- Source:
crawlStats(fn)
Returns crawled stats info.
Parameters:
| Name | Type | Description |
|---|---|---|
fn |
function | callback |
- Source:
getBucket(url, fn)
Get certain bucket for API.
Parameters:
| Name | Type | Description |
|---|---|---|
url |
String | |
fn |
function |
- Source:
getConfig(bot_name, fn)
Returns config of particular bot in the cluster.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
fn |
function | callback |
- Source:
getCrawledPages(bot_name, i, len, sort_key, sort_type, fn)
Returns crawled pages for API.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
i |
Number | offset |
len |
Number | length |
sort_key |
String | |
sort_type |
String | |
fn |
function | callback |
- Source:
getFailedPages(bot_name, i, len, sort_key, sort_type, fn)
Returns failed pages for API.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
i |
Number | offset |
len |
Number | length |
sort_key |
String | |
sort_type |
String | |
fn |
function | callback |
- Source:
getPage(url, fn)
Get certain crawled/uncrawled page for API.
Parameters:
| Name | Type | Description |
|---|---|---|
url |
String | |
fn |
function |
- Source:
getProcessedBuckets(bot_name, i, len, sort_key, sort_type, fn)
Returns processed buckets for API.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
i |
Number | offset |
len |
Number | length |
sort_key |
String | |
sort_type |
String | |
fn |
function | callback |
- Source:
getSeed(fn)
Returns seed data from cluster.
Parameters:
| Name | Type | Description |
|---|---|---|
fn |
function | callback |
- Source:
getTotalBuckets(bot_name, i, len, sort_key, sort_type, fn)
Returns total buckets for API.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
i |
Number | offset |
len |
Number | length |
sort_key |
String | |
sort_type |
String | |
fn |
function | callback |
- Source:
indexField(collection_name, index_name, fn)
Creates mongodb index for certain collection.
Parameters:
| Name | Type | Description |
|---|---|---|
collection_name |
String | |
index_name |
String | |
fn |
function | callback |
- Source:
readLog(bot_name, type, lines, fn)
Returns log data.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
type |
String | |
lines |
Number | |
fn |
function | callback |
- Source:
readTerminal(bot_name, fn)
Returns terminal data.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
fn |
function | callback |
- Source:
search(query, i, fn)
Text search on crawled data for API.
Parameters:
| Name | Type | Description |
|---|---|---|
query |
String | |
i |
Number | length |
fn |
function |
- Source:
setConfig(bot_name, js, fn)
Set config for particular bot in the cluster.
Parameters:
| Name | Type | Description |
|---|---|---|
bot_name |
String | |
js |
Object | |
fn |
function |
- Source:
setSeed(js, fn)
Set seed file for cluster.
Parameters:
| Name | Type | Description |
|---|---|---|
js |
Object | |
fn |
function |
- Source: