(private) new URL(url)
Returns JSON having url details
Parameters:
| Name | Type | Description |
|---|---|---|
url |
String |
- Source:
Methods
addAlternateUrl(l, lang)
Add alternate language urls
Parameters:
| Name | Type | Description |
|---|---|---|
l |
String | url |
lang |
String | language |
- Source:
getBucketId()
Get bucket id.
- Source:
getContentMd5()
Get md5 content.
- Source:
getDomain()
Get domain.
- Source:
getHTMLContent()
Get HTML content.
- Source:
getNutchKey()
Get nutch style url rep.
- Source:
getParent()
Get parent of this url.
- Source:
getParsedContent()
Get parsed content.
- Source:
getRedirectedURL()
Get redirect url.
- Source:
getResponseTime()
Get response time.
- Source:
getStatusCode()
Get status code.
- Source:
getURL()
Get url.
- Source:
getUrlId()
Get urlId.
- Source:
isAccepted()
Get accepted or rejected status.
- Source:
isIndexed()
Returns indexed status.
- Source:
isParsed()
Returns parsed status.
- Source:
setBucketId(idd)
Set bucketId
Parameters:
| Name | Type | Description |
|---|---|---|
idd |
String |
- Source:
setCanonicalUrl(l)
Set Canonical urls, which points the page having same content
Parameters:
| Name | Type | Description |
|---|---|---|
l |
String | url |
- Source:
setContent(content)
Set crawled content
Parameters:
| Name | Type | Description |
|---|---|---|
content |
Object |
- Source:
setContentMd5(l)
Set md5 hash of content this URL has.
Parameters:
| Name | Type | Description |
|---|---|---|
l |
String | string rep of hash |
- Source:
setFailedQueue()
Set failed queue
- Source:
setHeaderContentType(header)
Set header content type
Parameters:
| Name | Type | Description |
|---|---|---|
header |
String |
- Source:
setNormalQueue()
Set normal queue
- Source:
setParsed(content)
Set parsed content
Parameters:
| Name | Type | Description |
|---|---|---|
content |
Object |
- Source:
setRedirectedURL(url)
Set redirectUrl
Parameters:
| Name | Type | Description |
|---|---|---|
url |
String |
- Source:
setResponseTime(response)
Set response time
Parameters:
| Name | Type | Description |
|---|---|---|
response |
String |
- Source:
setStatusCode(code)
Set status code recieved on this url
Parameters:
| Name | Type | Description |
|---|---|---|
code |
String |
- Source:
setUrlId(code)
Set urlId
Parameters:
| Name | Type | Description |
|---|---|---|
code |
String | the urlId assigned by mongodb on insert |
- Source: