Extract vendor data from wowhead

[Loki]

I made a tool to extract data from the wowhead web. It is used basically to vendors that have a lot of items but it works with all.

http://www.mediafire.com/?v98b6b50i86j59o

The limitations are:

- extendedcost is always 0

- incrtime is always 0 on infinite items, for restockable items it is an hour, 3600 secs

- It only fills vendor_data table

Recommend to use only with vendors that sells only items that can be bought with gold. Otherwise you must fill the extendedcost column manually.

Instructions:

Go wowhead, a vendor like this:

http://www.wowhead.com/npc=2483

Right click, view source code. File menu, save as "npc.htm"

Put the .exe on the same folder that "npc.htm" and open it. You should get a file called vendor.sql. From the npc shown I get this:

-- Jaquilina Dramet'
SET @ENTRY := 2483;
DELETE FROM `npc_vendor` WHERE `entry`=@ENTRY;
INSERT INTO `npc_vendor` (`entry`, `slot`, `item`, `maxcount`, `incrtime`, `ExtendedCost`) VALUES
(@ENTRY,0,12250,1,3600,0),
(@ENTRY,0,2523,0,0,0),
(@ENTRY,0,2522,0,0,0),
(@ENTRY,0,2530,0,0,0),
(@ENTRY,0,25876,0,0,0),
(@ENTRY,0,2531,0,0,0),
(@ENTRY,0,12164,1,3600,0);

And this is what you get in the console:

http://imageshack.us.../dibujohbm.png/

The source code:

/*
Tool for parsing the Wowhead.com vendor pages
by Loki
*/

#include <iostream>
#include <fstream>

using namespace std;

int main(void) {

	ofstream myfile;
	ifstream readfile;

	myfile.open("vendor.sql");
	readfile.open("npc.htm");

	bool read=false;
	char raw;
	char name[50], vname[50];
	int ientry,slot=0, maxcount, incrtime, extendedcost=0; //extendedcost, slot, what?
	int avail, count1=0, count2=0, stack, ventry;

	while(readfile >> raw) {	//get npc name and id

		if(!read) {
			if(raw == 'I') {
				readfile >> raw;
				if(raw == 'n') {
					readfile >> raw;
					if(raw == 'f') {
						readfile >> raw;
						if(raw == 'o') {
							read = true;
						}
					}
				}
			}
		}

		if(read) {
			if(raw == 'I') {	//id
				readfile >> raw;
				if(raw == 'd') {
					readfile >> raw;
					if(raw == ':') {
						readfile >> ventry;
						cout<< "Npc entry: "<< ventry<< endl;
					}
				}
			}

			if(raw == 'm') {	//name
				readfile >> raw;
				if(raw == 'e') {
					readfile >> raw;
					if(raw == ':') {
						readfile >> raw;
							readfile.getline(vname,50,'}');
							cout<< "Npc name: "<< vname<< endl;
							read = false;
							break;
					}
				}
			}
		}
	}

	myfile << "-- "<< vname<< endl;
	myfile << "SET @ENTRY := "<< ventry<< ";"<< endl;
	myfile << "DELETE FROM `npc_vendor` WHERE `entry`=@ENTRY;"<< endl;
	myfile << "INSERT INTO `npc_vendor` (`entry`, `slot`, `item`, `maxcount`, `incrtime`, `ExtendedCost`) VALUES"<< endl;

	while(readfile >> raw) { //while for every item
		
		if(!read) {
			if(raw == 's') {	//read the correct piece of text
				readfile >> raw;
				if(raw == 's') {
					readfile >> raw;
					if(raw == 's') {
						read = true;
					}
				}
			}
		}

		if(read) {
			if(raw == ',') {	//get item id
				readfile >> raw;
				if(raw == '"') {
					readfile >> raw;
					if(raw == 'i') {	
						readfile >> raw;
						if(raw == 'd') {
							readfile >> raw;
							if(raw == '"') {
								readfile >> raw;
								if(raw == ':') {
									readfile >> ientry;
									count1++;
								}
							}
						}
					}

					if(raw == 'n') {	//get item name
						readfile >> raw;
						if(raw == 'a') {
							readfile >> raw;
							if(raw == 'm') {
								readfile >> raw;
								if(raw == 'e') {
									readfile >> raw;
									if(raw == '"') {
										readfile >> raw;
										if(raw == ':') {
											readfile >> raw;
											if(raw == '"') {
												readfile >> raw;	//ignore strange char
												readfile.getline(name,50,'"');
											}
										}
									}
								}
							}
						}
					}
				}
			}

			if(raw == 's') {	//get stacks
				readfile >> raw;
				if(raw == 't') {
					readfile >> raw;
					if(raw == 'a') {
						readfile >> raw;
						if(raw == 'c') {
							readfile >> raw;
							if(raw == 'k') {
								readfile >> raw;
								if(raw == ':') {
									readfile >> raw;
									if(raw == '[') {
										readfile >> stack;
									}
								}
							}
						}
					}
				}
			}

			if(raw == 'a') {	//get avail
				readfile >> raw;
				if(raw == 'v') {
					readfile >> raw;
					if(raw == 'a') {
						readfile >> raw;
						if(raw == 'i') {
							readfile >> raw;
							if(raw == 'l') {
								readfile >> raw;
								if(raw == ':') {
									readfile >> avail;
									count2++;
								}
							}
						}
					}
				}
			}

			if(raw == '}') {
				if(count1 == count2) {	//set maxcount and icrtime
					if(avail == -1 && stack == 1) {
						maxcount = 0;
						incrtime = 0;
						}
					else {
						maxcount = stack;
						incrtime = 3600;	//one hour, general cooldown
					}

					readfile >> raw;	//sql code out
					if(raw == ',') {
						myfile << "(@ENTRY," << slot <<"," <<ientry <<"," <<maxcount <<","<<incrtime <<","<<extendedcost <<")," <<endl;
						cout<< "Maxcount: "<< maxcount << " - Item: "<<ientry << " - "<< name<< endl;
					}
					if(raw == ']') {
						myfile << "(@ENTRY," << slot <<"," <<ientry <<"," <<maxcount <<","<<incrtime <<","<<extendedcost <<");" <<endl;
						cout<< "Maxcount: "<< maxcount << " - Item: "<<ientry << " - "<< name<< endl;
						read = false;
						break;
					}
				}
			}
		}
	}

myfile.close();
readfile.close();

system("PAUSE");

return 0;
}

Edited August 24, 2012 by DJScias
Fixed the link for mediafire

[thebillkidy]

Well publish the source code, then we need to know some other things from the devs and then i should be able to make this for mass fetching.

[Loki]

I posted the source code but it embarrass me a little. I have basic knowledge of c++ and I have done what I could. Basically, if wowhead does not change his structure this work correctly for what I said. But speaking about how the program is done I think that is very optimizable. If a pro read the code... How would you code the same without so many ifs (to recognize the wh code)? I am thinking about a function and a for/loop but maybe there is another way to do this.

[thebillkidy]

Well it aint hard to make it from an online source just the question is will it make any different? since i just heard that a lot of things are wrong.

[Loki]

Well it aint hard to make it from an online source just the question is will it make any different? since i just heard that a lot of things are wrong.

Well, "wrong" is not the question. It works so it is correct. But you can get working a program in many ways. I am trying to write it in the optimal way but it is not important for the purpose of the program.

Just try it and see that it do what it is suposed to do.

[thebillkidy]

Well it is great and i will certainly see how we can speed this up by the use of c# :D

[niotex]

I can't acces the link, but thanks for the source, copiled it myslef. Great tool for simple vendors.

[DJScias]

Fixed the link for mediafire, about time someone did. DJScias

[locario]

Would be great see the source code.

[Randy336]

Very nice release, this could be very helpful.

[meegosa]

link dead

[socra2]

Got a better one, supports extended costs but 434 only https://github.com/Sovak/WowheadRipper