Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If I was going to scrape this data and re-purpose it, I would've absolutely cleaned up those apostrophes. The pivoting between straight and curly would certainly be a pet peeve. Unless there's a semantic difference between the two I'm unaware of.


The semantic difference would be important in a song like Baby Got Back by Sir Mix-a-lot, which includes both speech quotes and imperial measurements.


Imperial measurements should use the prime symbol and not straight quotes: https://en.wikipedia.org/wiki/Prime_%28symbol%29#Designation...


The song was released almost a year before UTF-8 was first presented at USENIX, so I think it is reasonable for the lyrics to be expressed using the commonly available technology available of the time.


For lyrics, you should just transcribe the pronunciation, eg "six foot two".


I for one would impose metric... it may be harder to rap, but true talent steps up to a challenge...


"But I would walk 804 kilometres, And I would walk 804 more, Just to be the man who walks a 1,608 kilometres, To fall down at your door..."


It's so catchy! Perhaps: "16093.44 meters high" by "0.2286 meter nails"...


1.608 megametres :^)


The units are not spoken, but for clarity they should be notated in the lyrics.


TIL about prime symbols. Thank you!


My anaconda don't want none of your poor semantics.


I queried for this song's lyrics, and I found only a single quoting symbol (') used entirely throughout.

Taking a layperson's reading of the lyrics, I didn't find anything off due to quoting issues. If there was any disruption, it was minimal and unnoticed.


Sir Mix-a-lot is an American rapper so the measurements in the song are presumably in US customary system units and not imperial units.


Ahh, but is it US customary or US survey?


Could you try not to be so pedantic regarding my pedantry.


Yeah, makes sense, but this is still a pretty good approach. Inserting invisible or unusual Unicode symbols would prompt the scraper to carefully cleanup the read files (maybe even fixing these apostrophes as a result). Unusual whitespace is also likely to be removed and cleaned up. On the other hand, these alternating apostrophes have a chance to stay unnoticed (or neglected), falling through the cracks.


There is a semantic difference between the two. The straight quote is a superset of the curly one.

So "rock 'n' roll" is correct. And "rock ’n’ roll" is correct. But "rock ‘n’ roll" is not correct, since the wrong apostrophe is used. We're not quoting the letter n, we're showing that the letter a was removed.


The vast majority of content consumed by this scraper is never closely inspected by a human, though.


On this quantity of data, you wouldn't be able to do this manually.

If you hope to avoid being caught this way, I'm going to assume you noticed this without the benefit of hindsight and plan to correct all out-of-place Unicode characters automatically. How will you avoid over-correcting?

There's also no reason to believe this is the only fingerprinting Genius has done (they only need to publish the most obvious fail). For example, I can use the same fingerprinting technique but switch between American and British spellings.

This is not a straightforward problem.


I think if you had multiple corpuses of lyrics, you could cross-check for anomalies of any variety (odd quoting, switching between american/british english), etc.

The fingerprinting isn't likely applied to every song, to prevent obvious detection. If you went through multiple databases, you might see N prevailing copies of a song's lyrics, and 1 that seemed different. The one that's different has the anomaly.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: