Python to CSV or Excel Export multible data -
hey guys try write data csv file / excel.
the problem excel file don't work , csv shows me "element div @ xx.xxx.xx" etc. wan't line line product , price filled in right match. excel prefered. around 1000 product scrapped.
standard code test if code work:
from lxml import html import requests urls = ['http://whiskey.de/shop/aktuell/'] url in urls: number in range(1, 15): page_url = url + str(number) if number > 1 else url page = requests.get(page_url) tree = html.fromstring(page.text) prices = tree.cssselect('div#content div.price') names = tree.cssselect('div#content div.title a') print 'whiskey preis: ', [price.text price in prices] print 'whiskey names: ', [name.text name in names]
csv tested:
from lxml import html import requests import csv urls = ['http://whiskey.de/shop/schottland/'] url in urls: number in range(1, 10): page_url = url + str(number) if number > 1 else url page = requests.get(page_url) tree = html.fromstring(page.text) prices = tree.cssselect('div#content div.price') names = tree.cssselect('div#content div.title a') open('whiskey_test.csv', 'w') csvfile: fieldnames = ['preise', 'namen'] writer = csv.dictwriter(csvfile, fieldnames=fieldnames) writer.writeheader() writer.writerow({'preise': prices, 'namen': names})
there serveral issues here.
getting tags (div) text instead of writing tag elemnt itself
in "standart code" 'right' calling .text on selected elements.
print 'whiskey preis: ', [price.text price in prices] print 'whiskey names: ', [name.text name in names]
while here
writer.writerow({'preise': prices, 'namen': names})
you write tree-element fuile. instead like
writer.writerow({'preise': prices.text, 'namen': names.text})
however...
not writing several rows
assuming had datastructure contained price/name pairs (like list of tuples or dictionary) iterate of writing new line in dictwriter this:
for price, name in in productdictionary.items(): writer.writerow({'preis': price.text, 'name': name.text})
creating proper datastructure mapping prices names
now, dont think should scraping work :) basicly want use tree fetch corresponding pairs of price , name elements. once got dump them in dictionary or list of tuples. once got iterate on write csv shown above.
Comments
Post a Comment