david wong

Hey! I'm David, cofounder of zkSecurity and the author of the Real-World Cryptography book. I was previously a crypto architect at O(1) Labs (working on the Mina cryptocurrency), before that I was the security lead for Diem (formerly Libra) at Novi (Facebook), and a security consultant for the Cryptography Services of NCC Group. This is my blog about cryptography and security and other related topics that I find interesting.

Nodster day 2 posted April 2014

Okay. I'm not gonna talk too much about my new node-webkit project Nodster because I also have a lot of studies to do (exams next week!).

nodster

The problems

  • I'm still not streaming the mp3s properly. I think that I'll never be able to do it through the html5 audio and I need to look up other solutions. The do it yourself approach seems appealing and I'm gonna look into node speaker and node lame.
  • I'm using node google to crawl google and get links. It's not working properly and I'll have to dig into crawling google myself.
  • I'm parsing pages with a regex, it really seems to be slowing everything and I'm gonna look into using a dom parser. I heard about cheerio, is it good?

What does the app do?

  • Right now it's crawling google and avoiding useless websites like youtube, facebook, soundcloud, etc...
  • Then it goes one step deeper and looks for .mp3
  • It checks each mp3 for size (thanks to the headers) and download a bit of each to get metadata.
  • Displays the metadata and a link to play the file
  • If the link is clicked, the mp3 is downloaded to a buffer.mp3 and played.

The main problem is that it's slow, and it's not finding enough links. I could try to parse bing, yahoo and a list of mp3 finders. Or I could maybe try to optimize the requests to google...

Anyway, this app is bringing me a lot of problems to solve and it's pretty interesting :) some people have already forked it and someone has already requested a pull, so if you wanna help. Come and fork it!

Well done! You've reached the end of my post. Now you can leave a comment or read something else.

Comments

Thibaut

I have some node code to scrape google (that works) with captcha solving support (:p) I may push in on my github if you want to play with it.

As for decoding and playing mp3s, have you gave a look to AudioContext.decodeAudioData ? http://www.html5rocks.com/en/tutorials/webaudio/intro/

For parsing cheerio is not bad as far as I can tell, it works pretty well and perfs have not been a problem for me so far

Thibaut

Ping me on twitter if you want to talk

leave a comment...