how to counter TypeError: expected a character buffer object in my project

Vote count:

0

Hy so I made a simple web spider (I think) And while executing it i got the expected character buffer object error

import urlparse
import mechanize
import csv
from sys import argv
script, filename = argv
script, file = argv
script, input_file = argv
target = open(filename, 'w')
url = raw_input('Enter url \n>>') # Set the startingpoint for the spider and initialize
br = mechanize.Browser() # the a mechanize browser object

#create lists for the urls in que and visited urls
urls = [url]
visited = [url]

#cleaner
print "\n" *2000

# Since the amount of urls in the list is dynamic
# we just let the spider go until some last url didn't
# have new ones on the webpage
while len(urls)>0:
    try:
        br.open(urls[0])
        urls.pop(0)
        for link in br.links():
            newurl = urlparse.urljoin(link.base_url,link.url)
            #print newurl
            if newurl not in visited and url in newurl:
                visited.append(newurl)
                urls.append(newurl)
                print newurl
    except:
        print "error"
        urls.pop(0)

target.write(visited)
print "\nHere are the visited pages(you can find them in a txt file):"
print visited

So after reading a few things on the web I understood that you cant write a list to a txt file or something and I am out of solutions on what to use instead because I want to write all of my urls down but I do not what to have to copy past them all of the time... please help :)

asked 1 min ago

Lexostras Smith

1

how to counter TypeError: expected a character buffer object in my project