I can't stop coding this thing. What I've done today:
- This is the query I'm using in google now :
[your search] mp3 -facebook -youtube -soundcloud -last.fm -amazon -dailymotion -bleep
- I'm now parsing google and the following links with jQuery:
$(body).find('a') and then looking for good links with a regex. This is so much more quicker!
- There is still a part that is taking some time though, it's the metadata fetching. I still have to get every mp3 independently and download enough bits to learn its metadatas...
But it is working quite well as it is!
EDIT: I'm now fetching the metadata in parallel and it's super fast! I've also fixed a ton.
Okay. I'm not gonna talk too much about my new node-webkit project Nodster because I also have a lot of studies to do (exams next week!).
- I'm still not streaming the mp3s properly. I think that I'll never be able to do it through the html5 audio and I need to look up other solutions. The do it yourself approach seems appealing and I'm gonna look into node speaker and node lame.
- I'm using node google to crawl google and get links. It's not working properly and I'll have to dig into crawling google myself.
- I'm parsing pages with a regex, it really seems to be slowing everything and I'm gonna look into using a dom parser. I heard about cheerio, is it good?
What does the app do?
- Right now it's crawling google and avoiding useless websites like youtube, facebook, soundcloud, etc...
- Then it goes one step deeper and looks for .mp3
- It checks each mp3 for size (thanks to the headers) and download a bit of each to get metadata.
- Displays the metadata and a link to play the file
- If the link is clicked, the mp3 is downloaded to a
buffer.mp3 and played.
The main problem is that it's slow, and it's not finding enough links. I could try to parse bing, yahoo and a list of mp3 finders. Or I could maybe try to optimize the requests to google...
Anyway, this app is bringing me a lot of problems to solve and it's pretty interesting :) some people have already forked it and someone has already requested a pull, so if you wanna help. Come and fork it!
As always when I have to study for exams I find a gazillion other things to do.
Last night I downloaded node-webkit. I've been slacking when doing QT and WxPython because they were boring. And I thought that node-webkit would be as well.
One hello world later and I was amazed at how easy it was to create desktop application with it!
Spent my day trying to figure out what are pipes, streams, websockets... I have to admit I have a long way to go but I really enjoyed this session of node.js.
I had this idea of a free spotify that would just get mp3s from google. There are a lot of legal issues around so I'm wondering if I should really keep this one, but here is what I did today: Nodster (Node-webkit + Napster).
The code is not pretty as I was trying and discovering new things all along. The design is crappy but this is obviously because I had other better things to code first :)
I want to get the metadata of the mp3s to sort them more easily (most of them seems to have metadatas) but I have trouble figuring out how to do it.
Also it uses mp3skull.com to find mp3s, there are a lot others but eventually I wanted to just use google and crawl hard.
I could also add pictures for albums and... so many things to do!
If you want to help, fork away :)
So, the team behind OpenBSD has forked OpenSSL to create LibreSSL. They're rewriting code, they already found several bugs.
You can check what they do with some awesome journalism here.
I remember a time where people would advise to just hash the password with md5 before storing it into a database.
Then md5 became a bad choice because of the rainbow tables (precomputed tables of md5). The concept of salt helped (adding a secret value to passwords before hashing them).
But hash were never meant for encrypting passwords. As KDF. But KDF seems to be better a fit for that kind of task.
See Ty's blog post "please stop hashing passwords". He makes good points and advise using those following KDFs for the job:
Scrypt is the one used in Litecoin by the way.
I've read a LOT of stuff about procrastination. I have techniques:
I think those are all my techniques, I can't really think of any others.
A few months ago I started counting my calories intake, and I lost weight! It's magic. As soon as you start counting the bad stuff, you realize how much you're doing of it.
I've had this idea for quite a long time, a slack counter, that times how long you slack per day. It compares that to the average.
I don't know how to implement that, but a cellphone app would be the best suited I think. We always carry it, so we just have to launch the app and start a timer when we procrastinate. This + a firefox/chrome app that recognizes websites that are "time consumer" to add to your statistics. I'm sure that would help me work more!
When I'm really in trouble, and I can't seem to get motivated, I always take my iPod out and start a 30 minute timer. I do that, and then I find something to work on, anything, and I don't have to be efficient or know right away how I'm gonna work on it. I just have to do it, non-stop, to focus for just 30 minutes. And when the timer stops, I'm usually motivated enough to either keep going, or take a small break and start a bigger timer. I had the idea of implementing a 30 minute timer + a chain system, everyday you can start this 30 minute timer once and if you do work non-stop for 30 minutes while it runs, you will validate a day. Try validating the most days in a row!
Other techniques ?
This article brings 3 good points:
There are two ways to look at any task. You can do something because you see it as a way to end up better off than you are now – as an achievement or accomplishment. As in, if I complete this project successfully I will impress my boss, or if I work out regularly I will look amazing. Psychologists call this a promotion focus
when we say things like “I just can’t get out of bed early in the morning, “ or “I just can’t get myself to exercise,” what we really mean is that we can’t get ourselves to feel like doing these things.
Making an if-then plan is more than just deciding what specific steps you need to take to complete a project – it’s also deciding where and when you will take them.
If it is 2pm, then I will stop what I’m doing and start work on the report Bob asked for.
You have to let go
This article helped/helps me a lot. Every time I fall into a procrastination period (because it's a disease you'll have to live with for the rest of your life :D), I read that article.
I hadn’t figured out the skill that would save me from the procrastination.
Until I learned about letting go.
Learn from the pro
Katia Verresen is a "Mental Energy coach" for CEOs and founders. Here's a nice article about her and interesting stuff she has to say about being efficient, in the zone.
Verresen is a big fan of a tactic called “calendar blocking,” and she encourages her clients to identify the chunks of time on their calendars when they have the most physical energy for work.
The mood you wake up in is critical. Consider it to be the default font for your entire day.
For a marathon runner, people say that the day before and the day before that are the most important. If you mess up with your sleep one day, you'll feel the consequences until two days after.
Verresen actually tells her clients that if they haven’t gotten enough sleep, they shouldn’t take their emotions seriously the next day.
“Mental energy is the ability to separate yourself from your thoughts,” Verresen says. “Everyday, we have millions of thoughts that cause stress, anxiety, depression — that can stall you out in a million different ways — but you don’t have to believe them.”
“The best way to reset your mental energy is to get it up and out into the physical world,” Verresen says. “Screens don’t count. You should put whatever it is that’s bothering you or that you need to get done up on a white board. Write it down and stick it up on a wall.
I sometime stop what I'm doing to do the dishes. It clears my mind.
Verresen advises her clients to block off two slots ranging between 90 minutes to 2 hours every week as mental white space. They need to literally put it on their calendar and make sure they aren’t interrupted or disturbed during this time — ideally they shouldn’t have meetings right afterwards either.
I do this by doing my laundry. I take a paper and a pen with me and I go do my laundry for an hour.
It doesn't seem to be the only appearance of the Gröbner basis on the show:
In the Season 4 opening episode 'Trust Metric' (2007) of the television crime drama NUMB3RS, math genius Charlie Eppes mentions that he used Gröbner bases in an attempt to derive an equation describing friendship.
From wolfram alpha
I've run into this über cool explanation of the Fourier Transform thanks to mtodd's blog
Here's a bit from the introduction:
What does the Fourier Transform do? Given a smoothie, it finds the recipe.
How? Run the smoothie through filters to extract each ingredient.
Why? Recipes are easier to analyze, compare, and modify than the smoothie itself.
How do we get the smoothie back? Blend the ingredients.
And cool examples of what can be done with the Fourier Transform:
- If earthquake vibrations can be separated into "ingredients" (vibrations of different speeds & strengths), buildings can be designed to avoid interacting with the strongest ones.
- If sound waves can be separated into ingredients (bass and treble frequencies), we can boost the parts we care about, and hide the ones we don't. The crackle of random noise can be removed. Maybe similar "sound recipes" can be compared (music recognition services compare recipes, not the raw audio clips).
- If computer data can be represented with oscillating patterns, perhaps the least-important ones can be ignored. This "lossy compression" can drastically shrink file sizes (and why JPEG and MP3 files are much smaller than raw .bmp or .wav files).
- If a radio wave is our signal, we can use filters to listen to a particular channel. In the smoothie world, imagine each person paid attention to a different ingredient: Adam looks for apples, Bob looks for bananas, and Charlie gets cauliflower (sorry bud).
I'm in holidays for a week, easter I think, anyway, I didn't know what to do so I coded the Diffie-Hellman handshake, the ElGamal cryptosystem and the RSA cryptosystem in python.
You can check the code on github here: github.com/mimoo/crypto_studies
tests.py file to see how the classes are used. Here's an extract:
"""Testing Diffie Hellman
# 1. BOB
bob = DiffieHellman()
# G and g are generated automatically
print("G is a group mod %i and of order %i, and the generator g is %i" % (bob.G, bob.G, bob.g))
# We generate a secret and a public key
# 2. ALICE
# We already know G and g
alice = DiffieHellman(bob.G, bob.g)
# We generate the secret key and the public key
# 3. WE CREATE THE SHARED KEY
# Bob and Alice now have the same _sharedkey and the same public (G, g)
README says, it might be oversimplified and not totally correct. I mostly did that to do something in Python and also try to memorize how those systems work.
I've also done a lot of Unity this week-end. And also a bit of WxPython but I don't really like it. I think I should focus on QT and C++.
Those past few months I've been working on a C implementation of a whitebox using Chow et al's paper and the DES algorithm.
It's not done yet but the code is available on github. I also did a C implementation of DES just to get a grip on it, it's available on github as well.
I just did a presentation of the research my team and I did, I think it went pretty well. The slides are here
I've been looking for a summer internship and I haven't really found anything sor far. Although I've had some interviews with some start ups from the Silicon Valley (including TrueVault that really seemed like a good fit for a cryptographer in progress like me :D). But I've been unlucky so far since they're pretty busy, it's demo day-time for those applying to ycombinator there.
Anyway I still have 4 months of holidays this summer and I'm wondering what I'll do if I can't find anything in Mountain View (n_n I really want to go there).
If you know someone, or are interested in a passionate coder and eager learner, you can take a look at my resume here and rush to contact me before someone else does :)
Otherwise I'll spend more time coding personal projects and writing this summer (by the way, Korben, a famous influential blogger in France has written about me and my application 3pages.fr in a blog post. Huge amount of traffic in a few hours, 600 people signing up in a day. I envy his traffic.)
I've used Django for my last project and I found the documentation unclear and the list of things I had to do to code simple things and deploy were... a bit too much for a simple project.
I've glanced at the Flask documentation and have found it über-clear. The syntax seems to be pretty straight-forward as well. I'm really thinking about learning Flask for my next project and putting Django on hold. What do you guys think?
There's also a talk on web2py in the current PyCon. I don't know if it's for me but I really need something I can do quick prototypes on.
Sometimes I wonder if I should go back to PHP and try the new Laravel that really looks super cool :)
So I know how to use iptables, I know what a NAT is, but I don't want to learn how to exactly do it. Misery... I have to learn how to do it because I have an exam that will probably ask me how to do it in a few days. So I've been looking for a super simple tutorial, a 1 minute tutorial, on how to setup a NAT configuration with iptables in 1 minute. Couldn't really find it so here it is, if this is somewhat useful for someone, you're welcome.
For NAT to work, you have to allow forwarding on your server. Easy peasy:
$ echo 1 > /proc/sys/net/ipv4/ip_forward
Also, before adding new iptables rules, be sure to check what rules you already have
$ iptables -L
you should allow some forwarding for it to work (if the policy is default to DROP). But this not a tutorial about iptables.
I have a server with:
Let's modify the PREROUTING part. Traffic coming from internet on our public address (@pub) and trying to reach our machine:
$ iptables -t nat -A PREROUTING -d @pub -i eth0 -j DNAT --to-destination @priv
Let's modify the table nat, append a rule to the pretrouting section : something is trying to reach @pub ? Let's put it in our input interface eth0, jump to the Destination Nat protocol, which tells us to send the packet to @priv.
Now Let's modify the POSTROUTING part. Traffic coming from inside our network and trying to reach something, somewhere on internet:
$ iptables -t nat -A POSTROUTING -s @priv -o eth1 -j SNAT --to-source @pub
If the packet is coming from @priv, let's put it on our output interface eth1 and jump to the Source Nat Protocol that will modify the packet so it has the public address (@pub) as source.
Here! You did it. One private IP address mapped to one public IP address.
Same kind of configuration but now we have several private addresses and only one public address.
$ iptables -t nat -A POSTROUTING -s @priv/mask -j MASQUERADE
We can modify every packets coming from the subnetwork @priv to get masqueraded.
$ iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
Or we can just tell all the network to get masqueraded.
And this is it. No PREROUTING Needed.
Again, you're welcome ;)
A few hours after the start of the Heartbleed challenge, actually, just 3 hours after the start of the Heartbleed challenge. Fedor Indutny seems to have cracked it.
So now, chaos begins. If you own a certificate, you not only have to change it, but you also have to revoke it. I wonder how many will change, and how many will revoke.
You can check that he indeed did it by doing this:
Just to confirm it: put this into your /etc/hosts “184.108.40.206 http://www.cloudflarechallenge.com ” and visit “https://www.cloudflarechallenge.com/ “.
here why it works:
Putting that mapping in /etc/hosts lets your machine skip DNS lookup for that hostname, and just use his IP for that domain name.
Then, your browser checks the received certificate against the authenticated TLS connection, and sees that all is well, allowing you to connect without a warning.
Since the browser does not warn of a certificate mismatch, he must have a valid certificate for 'cloudflarechallenge.com'. QED.
The Cloudflare team reviewing the attack:
NSA is not happy. NSA is tweeting, tumblring (is this a verb?) and shouting loud and for all of who wants to hear it : they didn't know about the Heartbleed bug.
by the way they're talking about a "zero day" vulnerability, and now is a good time to learn what it is:
a so-called “Zero day” vulnerability because the developers of the vulnerable software have had zero days to fix it
I'm akin to trust them since... well. So many US websites were using OpenSSL and... it's not really nice if someone else eavesdrop on american citizen...
Anyway, this shows that the NSA has a long way to build trust again.