Sluggy NiftySearch

[DaveRhodes]

Ok, I promise this is a legit post and not one just to bump NiftySearch up to the first page of topics again...

So it turns out the Flag system was broken... it wouldn't flag strips, and even once I fixed that, it wouldn't display flagged strips first in the new unified transcriber.

But now it works, so if you see a problem in a transcript, flag away, folks!

I also removed all the flags on SGT's stuff. Most of those were fine, and didn't need to be flagged.

Now if I could only come up with a slightly nicer look for some of the pages...

[DaveRhodes]

*dredge*
Previously on Dave Posts About NiftySearch...

well shewt, looks like I'll have to rewrite the book-chapter-title parser for NiftySearch...
but I do like the new way to navigate straight to a date.

EDIT: YES! Thank you Emergency Pants, that b-c-t parser will be dead simple!

EDIT: YES!

Dam, Dave, that was fast!

EDIT: YES!

Dam, Dave, that was fast!

Oh no, that was me just rejoicing at my job being easier, not my job being done.

I'll probably be doing that by tomorrow.

(posted Saturday, August 1st, 2009)

So yeah, by tomorrow, I meant 3 weeks.
It's done now. I'm also storing the megatomes, but I haven't integrated them into the B-C-T pane yet 'cause that's already pretty crowded.

[DaveRhodes]

biggity-bump!

I'm totally not dead, I swear. And neither is NiftySearch. In fact, I just updated some code! Remember that spell checker thing I added to the search results so if you typed something in wrong, it would suggest what you might have meant?

Remember how it was horribly useless, because the algorithm tried to match (for instance) "Ailee" to "Aylee" by removing e, e , l and i, then adding y, l, e and e, counting each letter as a step off?

Well I finally got around to throwing out that algorithm (I didn't write it anyway... the only part of NiftySearch I didn't write) and use something else, the thing I should've used in the first place: Levenshtein distance! Turns out there's a function for it built into PHP, how nifty! It does a much better job at figuring out how many changes need to be made to a word to get it to turn into another word, and so I've revamped the search suggestions. Now it tends to spit out more at you (because I have it look a little further away just to make sure your intended word gets in there), but hopefully it presents you with the word you wanted.

For example, you can search for tork rife ailee zobmie, and, if you have the spell check enabled in Preferences, AMONG the suggestions will be torg riff aylee zombie!

Hopefully I can get my Google-fu on and make the spell checker automatically pick the best word... just as soon as I can have it determine what the best word is.

Also if the spell check doesn't tell you a clearly-wrong word is wrong, try searching for that word alone. The spell checker relies on what people have already entered into the transcriber, and it's possible a few typos got in. Simply flag the offending entries and you or someone else can correct them.

And as always, if you have any questions/comments/complaints/suggestions, post 'em please!

[swmartian]

Kudos, Dave, on getting a Sluggy Nifty Search ref link from Pete today (2/1/2010)! That's a nice pat on the back for a job well done!!

I've been cleaning out the transcripts; fixing typos, filling in blanks, etc. I hope that no one is offended that I've been homogenizing the 'style' of the transcripts as I go. I hadn't intended to when I started, but my anal retentiveness took over. If it makes you feel any better, I change my own transcripts, too. At any rate, I'm over half way done there -- nearly through with 2004...

[DaveRhodes]

(megabump)

Sup folks... Swmartian has been doing a wonderful job of sanitizing and locking all the transcripts, and a month or so ago, she actually cleaned and locked the entire archive!
And she found out that when the entire archive is locked, the transcriber goes nuts!
But today I took a look at it and found out what was wrong... it couldn't pick a date from the unlocked transcripts, since there weren't any... and it couldn't pick a date from the undone transcripts, since there weren't any... so instead it picked a blank date, which somehow already had a transcript attached. There was supposed to be a catch-all that was like "Well if I can't pick a date, then I'll just guess one at random." Buuuuut I had typoed it.

Fixed!

Now when you click the transcriber after all current strips have been transcribed and locked, you get a random strip and a lock icon. That's worth a bump, right?

Now I need to see if I can make the search results page look less... 1997.

[Arlo James Barnes]

*Approve*
The only disappointing thing for me is that most of the strips have been transcribed and locked already, so the chance of getting the few that have not been transcribed by selecting 'Random Strip' and pressing 'Go' is miniscule! EDIT: Just checked, all the strips have been transcribed. Problem solved.

[wade419]

Hello to all the sluggy info compilers! Hoping someone here might be able to answer the question of duplicate wikis. More info in this thread, please respond there if you guys have any ideas.

[DaveRhodes]

*Approve*
The only disappointing thing for me is that most of the strips have been transcribed and locked already, so the chance of getting the few that have not been transcribed by selecting 'Random Strip' and pressing 'Go' is miniscule! EDIT: Just checked, all the strips have been transcribed. Problem solved.

Hmm, if you ever see it give you a locked script while not all scripts have been transcribed, please let me know. It's not supposed to give you a locked script unless all of them are transcribed and locked.

As for the wiki question, I just answered that over there. Sorry for the confusion!

[swmartian]

Dave,

I also PM'd you on this -- but not sure where you'd see it first...

I'm not sure when this happened exactly -- I *think* it happened some time last week -- but at some point the transcript count got off. We are now 100.02% transcribed -- that is, we have one more transcript than comic????

[DaveRhodes]

Dave,

I also PM'd you on this -- but not sure where you'd see it first...

I'm not sure when this happened exactly -- I *think* it happened some time last week -- but at some point the transcript count got off. We are now 100.02% transcribed -- that is, we have one more transcript than comic????

Thanks for the heads-up... looks like some spammer tried to play the SQL injection game... managed to find a way to submit transcripts with empty dates. Not sure how that happened, but I'll look into it... the system cleansed it appropirately and dumped the garbage into the database, and since it was mostly gibberish and a broken URL, nobody would ever see it come up in the search. Ironically, once an empty-date entry is in the DB, no more can be created, so the spammer accidentally filled his or her own hole.

I deleted the entry, but now I'm tempted to just leave an empty date entry in the DB just to stop any more from getting in.

Sorry it took so long to respond, my spam filter must've caught the notification e-mails from sluggy.net.

[swmartian]

No problem. I figured you were busy or something. It didn't seem like it was a critical issue. Glad it wasn't too hard to track down. Thanks!

[DaveRhodes]

Poor Pete! I understand and respect his decision to go M-W-F only for a while, but he makes a good point: Most of the code associated with Sluggy assumes content every day. Niftysearch is no exception!

I'll work on adding something to blacklist days we know there isn't going to be any content (and hopefully make it be able to include last week and that one week back in August 2001) that's editable so when we start getting content on tuesdays and thursdays again, I can just throw a switch.

In the mean time, if you're transcribing and come across a date with no content, please just transcribe as usual, or if you're sure it's the no-content day, write "No Comic Today".

Unfortunately, the only way to check to make sure the untranscribed one is an empty date is by seeing the date in the URL of the input frame (visible by viewing source). I should probably stick a date up there if I don't get the blacklist system sorted out.

Also thanks to everyone who's been transcribing and administrating Niftysearch. Most of the times I've popped in to transcribe, everything was already done.

[swmartian]

I noticed that the Sluggy calendar is now fixed -- that is, you can no longer click on "off" days. Is there a way to tie into the calendar, perhaps?

Not that I have any problems filling in "No comic" on off days... Just asking... (and thanks for even bothering to think about it...)

(In response to NovemberMists, the calendar works on Firefox just fine)