app.Url.Url Class Reference

Inheritance diagram for app.Url.Url:

Collaboration diagram for app.Url.Url:

Public Member Functions
def	__init__ (self, url)
def	checkUrlCodeValid (self)
def	isValid (self)
def	getNormalized (self)
def	__lt__ (self, other)
def	__le__ (self, other)
def	__gt__ (self, other)
def	__ge__ (self, other)
def	__eq__ (self, other)

Static Public Member Functions
def	GetStats (urls)
	Parameters: urls - a list of Url instance objects to display stats for Returns: A list of stat mappings, a stat mapping has the following fields: source: the source url canonicalzed: the canonicalized url valid: whether the url is valid source_unique: whether the source is unqiue amongst the urls list canonicalized_unique: like source_unique but for canonicalized urls. More...

Public Attributes
	url

Detailed Description

Definition at line 6 of file Url.py.

Constructor & Destructor Documentation

__init__()

def app.Url.Url.__init__	(		self,
			url
	)

Definition at line 7 of file Url.py.

  def __init__(self, url):
    self.url = url

Member Function Documentation

__eq__()

def app.Url.Url.__eq__	(		self,
			other
	)

Definition at line 56 of file Url.py.

  def __eq__(self, other):
    return self.url == other.url

__ge__()

def app.Url.Url.__ge__	(		self,
			other
	)

Definition at line 53 of file Url.py.

  def __ge__(self, other):
    return self.url >= other.url

__gt__()

def app.Url.Url.__gt__	(		self,
			other
	)

Definition at line 50 of file Url.py.

  def __gt__(self, other):
    return self.url > other.url

__le__()

def app.Url.Url.__le__	(		self,
			other
	)

Definition at line 47 of file Url.py.

  def __le__(self, other):
    return self.url <= other.url

__lt__()

def app.Url.Url.__lt__	(		self,
			other
	)

Definition at line 44 of file Url.py.

  def __lt__(self, other):
    return self.url < other.url

checkUrlCodeValid()

def app.Url.Url.checkUrlCodeValid

(

self

)

Definition at line 11 of file Url.py.

  def checkUrlCodeValid(self):
    ret = True
    urlEncode = False
    subIndex = 0
    for index in xrange(0, len(self.url)):
      if urlEncode:
        if not (self.url[index] >= "a" and self.url[index] <= "f" or \
        self.url[index] >= "A" and self.url[index] <= "F" or \
        self.url[index] >= "0" and self.url[index] <= "9"):
          ret = False
          break
        subIndex += 1
        if subIndex == 2:
          subIndex = 0
          urlEncode = False
      if self.url[index] == "%":
        urlEncode = True
    return ret

Here is the caller graph for this function:

getNormalized()

def app.Url.Url.getNormalized

(

self

)

Definition at line 41 of file Url.py.

  def getNormalized(self):
    return url_normalize(self.url)

Here is the call graph for this function:

GetStats()

def app.Url.Url.GetStats (   urls )

static

Parameters: urls - a list of Url instance objects to display stats for Returns: A list of stat mappings, a stat mapping has the following fields: source: the source url canonicalzed: the canonicalized url valid: whether the url is valid source_unique: whether the source is unqiue amongst the urls list canonicalized_unique: like source_unique but for canonicalized urls.

Definition at line 71 of file Url.py.

  def GetStats(urls):
    stats = []
    canonicalized_list = [url.getNormalized() for url in urls]
    for url in urls:
      url_stat = {}
      canonicalized = url.getNormalized()
      url_stat['source'] = url.url
      url_stat['canonicalized'] = canonicalized
      url_stat['valid'] = url.isValid()
      url_stat['source_unique'] = (urls.count(url) == 1)
      url_stat['canonicalized_unique'] = \
          (canonicalized_list.count(canonicalized) == 1)
      stats.append(url_stat)
    return stats

isValid()

def app.Url.Url.isValid

(

self

)

Definition at line 31 of file Url.py.

  def isValid(self):
    ret = False

    regex = re.match(r'^((ht|f)tp(s?)\:\/\/|~/|/)?([\w]+:\w+@)?([a-zA-Z]{1}([\w\-]+\.)+([\w]{2,5}))(:[\d]{1,5})?/?(\w+\.[\w]{3,4})?((\?\w+=\w+)?(&\w+=\w+)*)?', self.url, re.IGNORECASE)  # pylint: disable=C0301

    ret = regex is not None
    if ret:
      ret = self.checkUrlCodeValid()
    return ret

Here is the call graph for this function:

Member Data Documentation

url

app.Url.Url.url

Definition at line 8 of file Url.py.

---

The documentation for this class was generated from the following file:

• sources/hce/app/Url.py