words about stuff
This post will be about a cool new feature engineering technique for encoding sets of vectors as a single vector - as described in the recent paper An efficient manifold density estimator for all recommendation systems. The paper focuses on EMDE’s applications …
In this post I explain why graph embedding is cool, why Pytorch BigGraph is a cool way to do it and show how to use PBG on two very simple examples - the “Hello World!” of graph embedding. All the code can be found here. With this you can quickly get started …
I recently came across several articles about failing data science projects (according to Gartner 85% big data projects are never fully productionised). The articles blame misaligned objectives, management resistance, unrealistic expectations, poor …
This is the second of a series of posts about things I wish someone had told me when I was first considering a career in data science. Part 1. For the purposes of this post I define a data analyst as someone who uses tools like Excel and SQL to interrogate …
This is the first of a series of posts about things I wish someone had told me when I was first considering a career in data science. Part 2 A popular meme places data science at the intersection of hacking, statistics and domain knowledge. It isn’t exactly …
TL;DR: I tested a bunch of neural network architectures plus SVM + NB on several text classification datasets. Results at the bottom of the post. Last year I wrote a post about using word embeddings like word2vec or GloVe for text classification. The …
TL;DR: I prepared 5 puzzles about statistics that should be accessible to anyone without being trivial. Scroll down for the puzzles. Data Science and Statistics “Data science is statistics on a Mac” “Data Scientist (n.): Person who is better …
This is a guest post by Javier Rodriguez Zaurin. My good friend Nadbor told me that he found on Reddit someone asking if data scientists end up doing boring tasks such as classifying shoes. As someone that has faced this problem in the past, I was committed to …
Trigger warnings: programming humor, algorithms and data structures, Java I’m interviewing data engineering contractors recently. All of the candidates are very senior people with 10+ years of experience. My go to question: Me: What data structure would …
This is the second post about my experiments with LSTMs. Here’s the first one. This is a great introduction by Karpathy. And this is an in depth explanation of the math behind. Python or Scala? Which should you use and when? Which should you learn first? …
In the last post I presented a way to do Bayesian networks with pymc and use them to impute missing data. This time I benchmark the accuracy of this method on some artificial datasets. Datasets In the previous posts I showed the imputation of boolean missing …
This is the first of two posts about Bayesian networks, pymc and missing data. In the first post I will show how to do Bayesian networks in pymc* and how to use them to impute missing data. This part is boring and slightly horrible. In the second post I …
I recently bought a deep learning rig to start doing all the cool stuff people do with neural networks these days. First on the list - because it seemed easiest to implement - text generation with character-based recurrent neural networks. watercooling, pretty …
In this post I will share some tips on the final aspect of datamatching that was glossed over in parts 1 and 2 - scoring matches. This is maybe the hardest part of the process, but it also requires the most domain knowledge so it’s hard to give general …
In the last post I talked about the principles of datamatching, now it’s time to put them into practice. I will present a generic, customisable Spark pipeline for datamatching as well as a specific instance of it that for matching the toy datasets from …
In this and the next post I will explain the basics of datamatching and give an implementation of a bare-bones datamatching pipeline in pyspark. ###Datamatching You have a dataset of - let’s say - names and addresses of some group of people. You want to …
I’ve had a lot of experience with tech interviews and recruiters in the past and I’m likely to have a lot more in the future. A non-negligible fraction of that experience ranged from “mildly annoying” to “exasperating”. …
I have blogged about the wide usefulness of topic models and I have benchmarked word-embedding-assisted classification on Reuter’s benchmark. This time I experiment with these ideas using a real world and decent sized dataset - the graph of UK/Irish …
I will share with you a snippet that took out a lot of misery from my dealing with pyspark dataframes. This is pysparks-specific. Nothing to see here if you’re not a pyspark user. The first two sections consist of me complaining about schemas and the …
In the previous post I talked about usefulness of topic models for non-NLP tasks, it’s back to NLP-land this time. I decided to investigate if word embeddings can help in a classic NLP problem - text categorization. Full code used to generate numbers and …
If you’re not primarily working with NLP you may not have been paying attention to topic modeling and word embeddings. In this post I intend to convince you that you should. ###Topic models Topic models are a set of models in NLP that discover common …
How do you train a binary classifier when you have only positive-labeled training examples? Impossible? Maybe. But sometimes you can do something just as good. Let’s start from the beginning… Lead generation Everyone and their mum in the b2b …
This is my first post. pls work pls This is code span (hopefully) def fibo(n): if n < 2: return 1 else: return fibo(n - 1) + fibo(n - 2)