User talk:NicDumZ

Archives for the NicDumZ talk page
	1, 2, 3

Run the DumZiBot script on a page

Request

Hey Nic, would you be able to run your DumZiBoT through the Öser page? Thanks. Khoikhoi 03:55, 23 March 2008 (UTC)

Dispenser just did it ;) NicDumZ ~ 11:10, 24 March 2008 (UTC)

DumZiBoT

Hallelujah! It's about time someone with the know-how stepped up to the plate. Next step: Create a version that works in real time after each edit. If not in real time, then "on-demand" for a particular article. The other day I corrected dozens of such links in 2008 NCAA Men's Division I Basketball Tournament. If I could have "submitted" that article to your bot for processing it would've saved me a boatload of time. davidwr/(talk)/(contribs)/(e-mail) 20:35, 24 March 2008 (UTC)

Dispenser ported my tool to http://tools.wikimedia.de/~dispenser/view/Pywikipedia !

Enjoy !

NicDumZ ~ 20:40, 24 March 2008 (UTC)

Awesome bot! Thanks for this. Tb (talk) 21:28, 24 March 2008 (UTC)

Thanks from me too. In the past I've done a lot of grunt work wrapping external links in ref tags, good to see someone following along and furthering that. :) Bryan Derksen (talk) 18:22, 27 March 2008 (UTC)

The da Vinci Barnstar

		The da Vinci Barnstar
		You are awarded this barnstar for enhancing Wikipedia by programming DumZiBoT, a reliable robot that has both greatly improved Reference lists and increased the productivity of Wikipedians. EconomistBR (talk) 21:32, 24 March 2008 (UTC)

Wooha !

Thanks, I appreciate it ;)

NicDumZ ~ 21:53, 24 March 2008 (UTC)

Chile

Please go through the Chile article again. I was forced to revert your edits as they were made on top of a vandalized version. Thank you. ☆ CieloEstrellado 02:04, 25 March 2008 (UTC)

Your edit got reverted.

Anyway, please consider using http://tools.wikimedia.de/~dispenser/view/Pywikipedia, it's great !

NicDumZ ~ 15:52, 25 March 2008 (UTC)

Question

Hello. I have a question if DumZiBoT can make one fairly simple task, it is quite similar to one he is already fixing so nicely. For details look here. Thank you very much. - Darwinek (talk) 15:39, 25 March 2008 (UTC)

Yes. Adding the refs tag is fairly easy : It's a common script from pywikipedia.

I have not even considered running it, for it is really easy to handle, and I thought that some other bot would already be doing it.

I'm afraid that running my bot on this would require another BRFA, tho.

NicDumZ ~ 15:45, 25 March 2008 (UTC)

Maybe you could create DumZiBoT2 or something like that, it would require another BRFA, too but it is not a matter of time. The work just should be done sooner or later. What do you think? - Darwinek (talk) 15:54, 25 March 2008 (UTC)

It might be better implemented into AWB (see WP:AWB/FR) as it does many other similar types of edits. Revision which implement <references/> appending. — Dispenser 23:07, 25 March 2008 (UTC)

Is there any option how to request this feature to be added to next version of AWB? - Darwinek (talk) 23:57, 25 March 2008 (UTC)

Minor Edits

Is there a way you could make the bot not mark its edits as minor when it modifies a certain number of bytes? It's a relatively minor thing but it came to my attention with this edit where DumZibot added over 2000 bytes but marked the edit as minor. Great bot, by the way! The Dominator (talk) 18:28, 25 March 2008 (UTC)

I could do that easily, but... what for ? xD

NicDumZ ~ 20:06, 25 March 2008 (UTC)

lol, good point, still it does help a little, for example when you're calculating the average number of minor edits or the percentage of edits that are minor, bots marking major edits as minor sort of alter the statistic. The Dominator (talk) 20:19, 25 March 2008 (UTC)

Ah. Would you get offended if I answered you "altering the statistic does not really justify the count of how many bytes are modified for each article" ? :D

Besides, the minor/normal edit limit would be very arbitrary, am I right ?

NicDumZ ~ 20:28, 25 March 2008 (UTC)

Grumble Link to Seroquel website

Hi. DumZiBoT recently generated a title for the link to the Seroquel home page on the quetiapine Wikipedia page. This was the change: diff. Prior to the change the link looked like this http://www.seroquel.com/. It was easy to tell the link was to a corporate web page. Now it looks like this Home. The revised link is not so clear. Perhaps you might modify the DumZiBoT parsing algorithm so that it generates a list of untitled links of the form “http://www.token.com/”. Some of these links may then be changed to the form “Token home page”, or the link may be left unchanged, or the old substitution algorithm might be used depending on the characteristics of the destination url. I have modified the link to look like this: Seroquel website. Regards. KBlott (talk) 20:39, 25 March 2008 (UTC)

I understand your concern. On de:, actually, they considered that problem as so important that it partly was the reason for asking me to stop DumZiBoT.

But actually, I disagree. Surely, and quite sadly actually, some webmasters don't really get the point of using a descriptive title for their pages. But Domain website as a title is really loosing some information :

1. There are more pages that have useful page titles than pages with undescriptive titles, and defaulting to a standard title for a minority of site is not the way out
2. Some domain names are actually pretty much uninformative : I'd better use a vague page title than an even more vague domain name.

I understand your concern, but I already thought a lot about that, and as of now I really think that the way DumZiBoT does is quite efficient, looking at how many websites use descriptive titles, and how many don't.

NicDumZ ~ 10:54, 26 March 2008 (UTC)

Actually, I agree with your proposition that page titles generally contain more information than domain names. On the other hand, it is easy to find exceptions to this rule. The problem is confounded by the fact that “information content” is essentially subjective. Ultimately any string in a grammar is meaningless, except in relation to the semantics that we as organics (or bots) may happen to assign to it. I think this problem is inherent to all sufficiently large parsing/editing tasks. I agree that web page designers often forget to give their web pages useful titles. In any case, the Seroquel link is now titled, so DumZiBoT will probably leave it alone from now on. Regards. KBlott (talk) 15:53, 26 March 2008 (UTC)

Recognition for a job well done a job well done another job well done

		The Working Man's Barnstar
		Every time I check my Watchlist, you and DumZiBoT have improved another article. Your continued work deserves recognition! TheRedPenOfDoom (talk) 03:20, 26 March 2008 (UTC)

Thanks! Great idea for a bot. — Omegatron 04:00, 26 March 2008 (UTC)

Great Bot. Yaki-gaijin (talk) 06:00, 26 March 2008 (UTC)

Thank you... ! :)

Really !

NicDumZ ~

Sinking_of_Prince_of_Wales_and_Repulse

Hi, Would you run the bot through Sinking_of_Prince_of_Wales_and_Repulse again please, It picked up that two off site referenced pages had poor page titles. I have now corrected these off site pages and if the bot could be run again it would reflect this on the above page references list. Apart from that, looks very interesting indeed. Nice one ;-) --Andy Wade (talk) 19:00, 26 March 2008 (UTC)

I'm glad my bot helped you improving your page titles.

However, as you can read in the [|User:DumZiBoT/refLinks|FAQ]], DumZiBoT only modifies untitled links, so it won't let me update the titles ;) (Or not with that current script) But you can easily =]

Cheers,

NicDumZ ~ 22:58, 26 March 2008 (UTC)

The pages in question are actually part of a frames site so normally they wouldn't show the page title.

But they were still untidy so I'm glad they're sorted out now. Cheers. --Andy Wade (talk) 23:39, 26 March 2008 (UTC)

Bad Title?

As seen in this dif you might want to add titles consisting solely of "test" to your title blacklist. Spiesr (talk) 20:18, 26 March 2008 (UTC)

Yes, you are right, actually. I added "test" as a blacklisted title, and DumZiBoT now runs with that updated list.

Thanks a lot for the report !

NicDumZ ~ 22:51, 26 March 2008 (UTC)

DumZiBoT's edit to Template:Infobox Planet

Your bot's edit to Template:Infobox Planet, [1], broke every page where that part of the template is used, by adding a references block to the end of the template. It might be an idea to restrict your bot to the main article space... Thanks. Mike Peel (talk) 20:39, 26 March 2008 (UTC)

Woops.

I'm truly sorry for that. I brought the bot to a stop, changed the references block code to add it only if it's in the main article namespace.

It still however add title to the references in every namespace, because I don't see any problem with that (Or please tell me if you can think of any)

Thanks for the kind report.

NicDumZ ~ 22:46, 26 March 2008 (UTC)

Good title

Sorry but nice name for your bot. I am not exactly sure why it labelled a particular report as Microsoft Word. How exactly is the bot meant to work? Shouldn't it read the title that is inside the document? here Simply south (talk) 21:32, 26 March 2008 (UTC)

Well, what is the title that is inside the document?

<dc:title><rdf:Alt><rdf:li xml:lang='x-default'>Microsoft Word - LIP Chapter 3 - Boro Policy Statement _2_.doc</rdf:li></rdf:Alt></dc:title>

... I cannot quite fault the bot here either. :) — the Sidhekin (talk) 21:41, 26 March 2008 (UTC)

For one thing, it is not a .doc but a .pdf (and where did that code come from, maybe i am in way over my head somehow) here Simply south (talk) 21:54, 26 March 2008 (UTC)

That code came straight out of the PDF file. Just read it in a text editor instead of a PDF viewer, and you'll find it. At least part of it are human-readable. :) (I'm betting the PDF file has been created from a .doc file, and no one thought to give it a proper title.) — the Sidhekin (talk) 22:11, 26 March 2008 (UTC)

... or, if you have the right tools:

sidhekin@blackbox[23:13:02]~$ pdfinfo lip_chapter_3_-_boro_policy_statement_.pdf 
Title:          Microsoft Word - LIP Chapter 3 - Boro Policy Statement _2_.doc
Author:         
Creator:        PScript5.dll Version 5.2
Producer:       Acrobat Distiller 6.0 (Windows)
CreationDate:   Tue Oct  3 13:23:58 2006
ModDate:        Tue Oct  3 13:23:58 2006
Tagged:         no
Pages:          22
Encrypted:      no
Page size:      595 x 842 pts (A4)
File size:      189739 bytes
Optimized:      yes
PDF version:    1.4
sidhekin@blackbox[23:13:20]~$ 

(It may even be your PDF viewer displays the title; check the window title bar or document info dialog?) — the Sidhekin (talk) 22:16, 26 March 2008 (UTC)

Thanks a lot, Sidhekin, for your help here, you got it perfectly right ;) NicDumZ ~ 22:55, 26 March 2008 (UTC)

Ah ?! How is "DumZiBoT" a nice name ? Out of inspiration, I just used the last part of my username, which derivates from my real name have I unwillingly made it sound... special ? :) [I'm not a native English speaker and sometimes just miss these little things] NicDumZ ~ 22:55, 26 March 2008 (UTC)

Erm... sorry.. unfortunately "dum" (with a b at the end) means stupid. I was not meaning anything bad. It is just a funny and rather ironic name. It is doing good work so... never mind.

Computer language is complete gibberish but i have found the part you are referring to. Simply south (talk) 23:04, 26 March 2008 (UTC)

Ah xD I knew that but didn't quite made the link. Don't worry, I really don't feel offended by that coincidence ;) NicDumZ ~ 23:09, 26 March 2008 (UTC)

Thanks for DumZbot!

The bot is doing a great job. Having said this, once it gets through Wikipedia, it would be nice if it ran "periodically," and not every night maybe? My watchlist gets flooded with entries which are all improvements, I'll say that. And yes, we should have done it right the first time and this wouldn't be happening.

Would once a month be too infrequent once the whole encylopedia is processed? Again, a brilliant job whoever thought of this. Has been needed for a long time! Student7 (talk) 23:16, 26 March 2008 (UTC)

Thanks for your Thanks ! :)

You may imagine that searching for the pages that contains bad references is a heavy process. (If not, I'm telling you ! :] ) Since only a few pages, compared to 2 millions of articles, need fixing, DumZiBoT is not retrieving all the pages from the servers just to alter the ones that need alteration. I use XML dumps for my work.

And actually, the en: database is only dumped every two months. DumZiBoT is working on the dump of March, 15th, but last available dump, (and last run) is from January, 9th.

Yes, as of now, DumZiBoT is only fixing the bare references inserted during that two month timespan : It takes quite a long time :) (DumZiBoT has been continuously running for nearly 80 hours now and I don't really know when it'll stop !)

Now, I'm not sure that I could improve that behavior. Knowing the dumps issue (I only get to know the list of articles needing modification every two months or so) what would you suggest me to do ?

Thanks,

NicDumZ ~ 23:53, 26 March 2008 (UTC)

I think you are telling me that, in the future, you will essentially run this "every two months." I appreciate that it has run for several days on January's list and will continue many more days until through. I vote for that! I assume that future runs might be shorter. But either way, assuming I'm looking at an "average sample" I won't see more updates than I'm looking at now. And maybe a lot less once it is through. Yes, it needs to keep going. A real plus for references. Editors are encouraged to update the quality of some references now that they have names. It should help a lot. Student7 (talk) 00:49, 27 March 2008 (UTC)

PDF being called Microsoft Word

http://en.wikipedia.org/w/index.php?title=MPEG_transport_stream&diff=201201845&oldid=199427990 The first reference it titled a PDF document microsoft word. Can you do something about that? Daniel.Cardenas (talk) 01:44, 27 March 2008 (UTC)

see Good title just above ;)

NicDumZ ~ 07:03, 27 March 2008 (UTC)

Are you going to do something about it? A PDF is not microsoft word. I suggest you add a rule to do nothing in this case. Daniel.Cardenas (talk) 11:23, 27 March 2008 (UTC)

A fix may be to check if titles have .foo in the end, and if they do, remove it. Martijn Hoekstra (talk) 07:59, 28 March 2008 (UTC)

Japanese characters

In this edit the bot generated a weird title for the ref. I'm guessing it is because the page is in Japanese? Just thought I would let you know. -- Ned Scott 06:54, 27 March 2008 (UTC)

Thanks !

It was not because the title was in japanese, it was because no encoding is actually specified in the HTML source : It is not standard complying, and there is no precise way to tell which encoding it is (My Firefox fails at outputing correctly the page)

However, I improved a bit the rules on .jp domain names, and it's better now.

Thanks again ;)

NicDumZ ~ 07:16, 27 March 2008 (UTC)

Request running of bot?

Is there any way we can request running this bot on a page? Like maybe putting a tag at the top or something?--Paul McDonald (talk) 12:55, 27 March 2008 (UTC)

See http://tools.wikimedia.de/~dispenser/view/Pywikipedia which is an online version of the bot. — Dispenser 14:13, 27 March 2008 (UTC)

Bug report

Hi there! I remember pointing out to you once that the bot does not always handle titles in Russian correctly, and I think you've fixed the problem for the most part, but today this edit popped up in my watchlist, and it seems to have the same problem. Don't know if it's something wrong with the site or with the bot, but I think it's worth a closer look. Cheers,—Ëzhiki (Igels Hérissonovich Ïzhakoff-Amursky) • (yo?); 14:36, 27 March 2008 (UTC)

Well, yes, again, it's a website that does not give out any encoding. :( Added a rule for .su websites, and it works

Thanks for the report ! :)

NicDumZ ~ 07:50, 28 March 2008 (UTC)

Actually no, it doesn't. The link you provided shows gibberish in Cyrillic letters. Looks like wrong encoding was selected, and it looks like KOI8-R was involved at some point in the decoding process. Sorry for the bad news!—Ëzhiki (Igels Hérissonovich Ïzhakoff-Amursky) • (yo?); 13:56, 28 March 2008 (UTC)

Better then. DumZiBoT now use KOI8-R instead of windows-1251 as a default Cyrillic charset. But actually, since both of these encodings are 8 bits, *any* text could be decoded using KOI8-R. What I'm saying here is that I've just switched priorities : If someone here comes and tell me that some title ot decoded as garbage since DumZiBoT used KOI8-R where windows-1251 would have been better, I won't be able to do anything more...

NicDumZ ~ 16:38, 30 March 2008 (UTC)

DumZbot reference conversion style "problem"

Currently 'DumZbot' converts:

<ref>http://www.energybulletin.net/2389.html</ref>

Into:

<ref>[http://www.energybulletin.net/2389.html Europe Worries Over Russian Gas Giant's Influence | EnergyBulletin.net | Peak Oil News Clearinghouse<!-- Bot generated title -->]</ref>

I think the idea is great, but the destination format isn't really that good. I would suggest the following format:

<ref name=wwwenergybulletinnet2389html>{{cite web|title=Europe Worries Over Russian Gas Giant's Influence | EnergyBulletin.net | Peak Oil News Clearinghouse|url=http://www.energybulletin.net/2389.html}} 080327 energybulletin.net</ref>

(dateformat is YYMMDD)

That way one can see 1) which site the information comes from without clicking on the link 2) date when the link was retrieved 3) Any updates to the template can utilise the collected information structurly 4) the reference can be used several times.

Otherwise all is well :-)

Electron9 (talk) 18:49, 27 March 2008 (UTC)

Here is your answer.

Thanks ! :)

NicDumZ ~ 15:44, 31 March 2008 (UTC)

Great Page!

Really nice job on your page. I like it, so keep up the good work! =) --Cher <3 (talk) 00:24, 28 March 2008 (UTC)

User:DumZiBoT ? :) NicDumZ ~ 07:45, 28 March 2008 (UTC)

conversion from bare ref, causes the reference to be listed multiple times

I see you have edited couple of references in article Dada Kondke. These changes have lead to the references section below to be populated with same title multiple times. Is there a way to fix this? Appreciate the conversion process but if the problem persists then its annoying :) --Kedar (talk) 06:53, 28 March 2008 (UTC)

Well, just don't use thrice the same reference, name it :)

See Wikipedia:Footnotes#Naming a ref tag so it can be used more than once

NicDumZ ~ 07:44, 28 March 2008 (UTC)

Suggestion

I think this bot is a great idea and it works perfectly as far as I've seen. I just have one suggestion: I find that the link text this bot generates is actually less descriptive than the URL. The bot just uses the title of the page, which usually doesn't distinguish the link all that well from others in the reflist. For example, if there's an article on Joe Smith, sourced with a few different biographies on that person, the links would all read "Joe Smith bio", "the life of Joe Smith", or something similar. There isn't much to distinguish one from the other, especially as far as which are from reliable sources.

The most important thing about references isn't really the title of the page, but the root site they're located on. I wonder if you'd consider modifying your bot to include the root site address in addition to the page title -- for instance, something like "Title at Site.com" (Joe Smith bio at timemagazine.com). This would allow a casual glance of the reflist to reveal any unreliable sources, any glaring omissions of sources that should be there, etc.

Thanks and please let me know your thoughts. Equazcion •✗/C • 23:00, 28 Mar 2008 (UTC)

Dumzibot: heaven or hell?

Bot made another error. Can't you people just do the links your selves?

[2]

Death Valley? izaakb ~talk ~contribs 00:10, 29 March 2008 (UTC)

Well. Try. Open the pdf. Title ?

Being aggressive when submitting an invalid bug report is just... ridiculous.

NicDumZ ~ 04:12, 29 March 2008 (UTC)

How 'bout answering the polite and accurate ones then? Equazcion •✗/C • 05:07, 29 Mar 2008 (UTC)

No. I answered that annoying question at 5 am local time, because it was just too much. But I need to think more about serious questions to issue an adapted answer :) NicDumZ ~ 09:49, 29 March 2008 (UTC)

The correct name of the PDF is Lethal Lou's: Profile of a Rogue Gun Dealer not "Death Valley". What PDF is called "Death Valley?" Not at the link below:

http://www.gunlawsuits.org/xshare/pdf/reports/lethal-lous.pdf Death Valley -- Bot generated title --

rgds izaakb ~talk ~contribs 21:35, 29 March 2008 (UTC)

The correct name of the PDF notwithstanding, the actual title of the PDF is "DEATH VALLEY". Observe:

sidhekin@blackbox[22:43:15]~$ pdfinfo lethal-lous.pdf 
Title:          DEATH VALLEY
Author:         vice
Creator:        Acrobat PDFMaker 7.0.5 for Word
Producer:       Acrobat Distiller 7.0.5 (Windows)
CreationDate:   Tue Sep  5 13:44:38 2006
ModDate:        Tue Sep  5 14:02:06 2006
Tagged:         yes
Pages:          25
Encrypted:      no
Page size:      612 x 792 pts (letter)
File size:      3643054 bytes
Optimized:      yes
PDF version:    1.6
sidhekin@blackbox[22:43:43]~$ perl -nle 'print if /title/.../title/' lethal-lous.pdf 
         <dc:title>
            <rdf:Alt>
               <rdf:li xml:lang="x-default">DEATH VALLEY</rdf:li>
            </rdf:Alt>
         </dc:title>
sidhekin@blackbox[22:43:46]~$ 

Computer programs don't generally go around inventing titles. — the Sidhekin (talk) 21:44, 29 March 2008 (UTC)

I been looking at the bot as doing work for editor who have been too lazy to use half proper references. Even if the titles are so great they are very valuable in dead link recovery. — Dispenser 06:59, 30 March 2008 (UTC)

Excessive link text

The bot generally does a good job, but look at this diff. Perhaps it should have some maximum text length? ✤ JonHarder ^talk 13:37, 29 March 2008 (UTC)

Ah, thanks a lot for the report !

Actually, I remember being asked about that problem, but can't remember why I did not fix it :(

Titles longer than 250 characters (arbitrary length) are now skipped.

NicDumZ ~ 16:17, 30 March 2008 (UTC)

If you skip, then please chop like this: "A really long line that is truncated" -> "A really long line that is tr.."

Rather than doing "A really long line that is tr", such that readers will know it's truncated. Electron9 (talk) 21:20, 30 March 2008 (UTC)

Character encoding issues

From pywikipedia's BeautifulSoup.py

try:
    import chardet
#    import chardet.constants
#    chardet.constants._debug = 1
except:
    chardet = None
chardet = None

Look at the last line. Web version updated, sources at http://tools.wikimedia.de/~dispenser/resources/sources/ — Dispenser 22:13, 30 March 2008 (UTC)

Ah, right

Good catch, it might help a lot ! (Sad that the run on the current dump is over, tho !)

I will test how chardet can improve DumZiBoT behavior.

I will commit that fix in pywikipedia trunk, as soon as I'll be able to.

NicDumZ ~ 15:48, 31 March 2008 (UTC)

Thanks

Thanks for converting the source links in relations to the PS2 version of Syphon Filter: Logan's Shadow. I usually have trouble with those. Beem2 (talk) 04:01, 31 March 2008 (UTC)

bot spelling error

Your bot seems to have misspelled "Association" quite a few times. I wonder if you could rerun to fix the error. A specific example is here, and a search shows it's fairly common. Regards, Jpmonroe (talk) 07:14, 31 March 2008 (UTC)

Yeah, basically, the title from the ARIA website is misspelled.

The website is wrong, not my bot, which just copies the title from the website.

NicDumZ ~ 11:16, 31 March 2008 (UTC)

yet another feature request - cite web templating

I'm sure you must've been asked this often; apologies (and just ignore this) if so... but could (perhaps the next revision of) DumZiBot convert the labelled URLs to web cite templates? This would mean that it could put in the retrieved date and would make it easier for other editors to flesh out the refs with richer information later. I appreciate all the work you and your bot have done! Pseudomonas_(talk) 10:02, 31 March 2008 (UTC)

Yes, I've been asked that often. However, a few minutes ago, my FAQ page was not answering that FAQ, so don't apologize ;)

Here is your answer.

Thanks ! ;)

NicDumZ ~ 15:43, 31 March 2008 (UTC)