A new website

So I have a new website… You may be wondering why I did that considering that I rarely update this website? Well, basically I decided I didn’t want to get rid of this website and, at the same time, I didn’t want to pay Squarespace prices to maintain it. So I moved to the slightly cheaper WordPress and let’s see how it works.

Don’t get me wrong, I really like Squarespace. It was easy to use, had way more intuitive navigation and formatting, and it did make my website look better without a lot of effort. So I do feel a little bad that I left, but I couldn’t convince myself to pay almost $200 a year to keep a website that I’m not using.

So, do I have plans for this website? I’ll get back to you on that. I do want to write more and share my ideas more often, but right now I’m not convinced I have the time to do it. Maybe 2020…

Classical Music Streaming

It has been observed by many people that “traditional” streaming services are terrible at classical music.

PCmag: Primephonic Wants to Save Classical Music, 1 Stream at a Time

Forbes: Meet Primephonic, The Streaming Company On A Mission To Save Classical Music

LifeHacker: The Best Classical Music Streaming Service Is Idagio

TechAcute: IDAGIO: The Streaming Service For Classical Music Fans

And they are pretty terrible. Classical music is complicated… There are lots of dimensions in which you can slice and dice the music, because the stronger dissociation of composer and performer, and the greater diversity of well-defined genres. I’m not saying that popular music doesn’t have as much diversity (although it might be true that it doesn’t), just that the taxonomy is not as well developed. Another dimension of complexity is the fact that a piece of classical music often needs to be taken as a complete unit, even though it might be split into multiple tracks (but not always, and not for everybody).

So enter the latest contenders, Primephonic and Idagio. How do they stack?

TL;DR: they are certainly better at handling classical music than the “standard” streaming tools. But they are far from perfect. And their “imperfection” is inconsistent. IDAGIO has a big defect (at least on the iOS client) of not supporting today gapless playback, so between tracks you get this short gap which is very annoying. This makes Primephonic the winner in my opinion, but not by much.

I started a trial on each and decided to do some comparisons for how I would use it. Here are the use cases that I tested:

  1. Query for a standard piece with an expected high number of recordings

  2. Query for a standard piece with less recordings

  3. Query for a series of pieces with low number of recordings

  4. Query for ensemble

I also discussed a couple of features related on how it is to listen to long pieces, make playlists and discovery.

I did not go into sound quality. That requires a fair amount of time investment in an environment where I could listen to the quality, so I just accepted what they advertise (Primephonic 24bit FLAC and IDAGIO 16bit FLAC) and that probably if you want to pay extra for it (for both you pay extra for lossless), you will get reasonably equivalent quality which will be more limited by the source recording than the streaming format.

1. Query for standard piece

beethoven string quartet 14

beethoven string quartet 131

(refers to Ludwig van Beethoven’s String Quartet No. 14 in C# minor op. 131)

IDAGIO:

“131” returns two works

  • String Quartet No. 14 in C sharp minor op. 131 (arr. for Piano) – single recording

  • String Quartet No. 14 in C sharp minor op. 131 – 53 recordings

Oddly, when I was typing it and was at “beethoven string quartet 13” it showed another work

  • String Quartet No. 14 in C sharp minor op. 131 (Arr. for String Orchestra) – 5 recordings

“14” returns:

  • String Quartet No. 14 in C sharp minor op. 131 (arr. for Piano) – single recording

  • Sonata for Piano No. 9 in E major op. 14/1 (Version for String Quartet)

This inconsistency made me ready for what was coming next for the next searches.

Primephonic

“131” returns 0 works! But quite a few albums. That’s what made me decide to also try “14”.

“14” returns 1 work:

  • String Quartet No.14 in C-sharp minor – 93 recordings, but when you click on it it only shows 82 recordings mixed orchestral and quartet. It had a lot of duplicates, but seems to have more recordings than IDAGIO.

2. Query for a standard piece with less recordings

“part festina lente”

for Arvo Pärt Festina Lente

IDAGIO

Not marked as “works” but returns “Festina Lente, for Harp and String Orchestra (1986, rev. 1990) – 5 recordings

Primephonic

“Festina lente” – 8 recordings

Interestingly the recordings have very little overlap.

3. Query for a series of pieces with low number of recordings

“bachianas”

Heitor Villa-Lobos “Bachianas Brasileiras No. 1-9”

IDAGIO

  • Shows (out of order) Bachianas Brasileiras 1-8 with dates

  • In the bottom has repeated 1 & 5 (twice)

Primephonic

  • Shows (out of order) Bachanas brasileiras 1-9 but missing 6

  • Repeated 4 & 5

4. Query for ensemble

“roomful of teeth”

IDAGIO

  • Only had one match for Berio: Sinfonia

Primephonic

  • 3 albums

  • Does have the Berio above, but doesn’t show in the search results (although it is in the metadata)

Other use flows

Recordings

They both have a concept of recordings which is a pair of work+recording, which allows you to add to playlist all movements for a single piece easily. That does create sometimes a little bit of a confusing point of what you are looking at when navigating. Moreover, on IDAGIO they have a concept of a “collection” and you can add “tracks”, “recordings” and “albums” to the collection. Some albums contain a single work, so sometimes you may think that you are adding a “recording”, but it ends up in the “album” because you were actually at the album view.

I think it’s a necessary concept, I just don’t think they cracked the UI component of it.

Playlists

I have this odd use case of creating a “today” playlist, where I accumulate the pieces that I want to listen to today. “Today” doesn’t mean that I’ll be able to listen to all those pieces in a single day and that’s where I have an unsolved issue: I’d love to know the last piece that I listened to on my playlist. Spotify (before my account started being shared with other people in my household) was pretty good at giving me where I was when I left, so, unless I switched to another playlist/album, I could just continue where I left off. Neither IDAGIO or Primephonic seem to do that. Often when I open their app, I start from scratch and I have to figure out where I was. I have a similar note below on playback.

Beyond that, playlists work as expected. You can add tracks, albums or recordings to them. But if you are on an album and you want to add a recording from that album (i.e. multiple tracks at once for the same work), you have to first navigate to the “recording” view for those tracks and then add them to your playlist.

Playback

This is probably one of the weakest points on IDAGIO, which made me decide that today (when I tested it) they are not the winner: it doesn’t support gapless playback. And that’s a big sin especially for classical music where you have a lot of works that have multiple movements that sometimes, by design by the composer, don’t have a break between them. Now on playback IDAGIO adds a break.

Beyond that, I also felt that IDAGIO took a little longer to start playing than Primephonic.

Finally, related to the “starting from scratch” that I mentioned on playlists, I also can never get it to autoplay when I start a new bluetooth connection. For example, my car connects to my phone on bluetooth and then I can play music through the car sound system, like most cars today. Let’s say that I select something on IDAGIO or Primephonic to play. Then I park the car, go get something and then when I get back to the car, I expect the bluetooth to connect again and trigger play on where I was, but that never worked on my tests for any of those two services. Kind of like the “start from scratch”, they just don’t know what to play. I have to go to the app and play from there.

Other Notes

For sure IDAGIO has the most number of features from the two:

  • Music by mood

  • Good music by musical instrument with a very large selection of instruments

  • More selection on pre-built playlists

But I don’t think that’s really enough to make it the winner, but, in technical terms, I believe the gapless playback is probably a smaller missing feature than the features above, so it might not take very long for IDAGIO to be my recommendation.

Ph.D. flashback of the day: scale-free networks

I was reading an article today from Quanta Magazine: Scant Evidence of Power Laws Found in Real-World Networks which refers to a posted article in arXiv: Scale-free Networks are Rare by Anna D. Broido and Aaron Clauset. And that gave me flashbacks of my Ph.D…

While, scale-free/power-law distributed networks weren’t really the main focus on my research, it did influence what I was doing, as it related to graph-structured databases, where a lot of that structure exists and affects scalability of your analysis. But, more importantly, it influenced a collaboration that I had with another researcher on the same department, Steve Morris. His actual interest was really on power-law distributed networks and his belief was that there was signal to be observed from when a network deviated from being power-law distributed.

During one Summer, we sat together and decided that the way researchers were claiming that everything was power-law distributed was by plotting it in a log-log scale and drawing a line through the points. We hypothesized that it was not a very good way to show that it followed the right distribution and we should have an actual statistical test. My proposal was to use bootstrapping and the Kolmogorov–Smirnov test. So we co-wrote a paper on it: Problems with Fitting to the Power-Law Distribution.

We didn’t have as much data to play with as the paper that I mentioned above, but we also did conclude that almost nothing was actually power-law distributed. And, until this date it’s the paper that I wrote with the most number of citations (622, according to Google Scholar).

We were onto something back then… Oh, well…

AI stigma

The other day I was noticing how many things out there tout to be using AI to solve things today and that’s a great thing. That gave me flashbacks of when I started at Amazon, in 2004. Back then I was just finishing my Ph.D. in a machine learning area, dealing with feature extraction on graph-structured databases, keeping a “fond” eye for the future of the web, the Semantic Web, where computers would be able to interact with the web as well as humans and that would be the turning point for what we’ve imagined as AI back then.

But we didn’t call it AI. AI wasn’t seen as a very good term. We talked about expert systems, or sometimes we did mention machine learning. But AI evoked two negative reactions:

  1. From the general population, AI was like 2001’s HAL 9000 or the Terminator, a robot/computer that was going to kill us all.
  2. For the research, and especially professional population, AI was that dream that people had in the past that just didn’t work. But “expert systems” were showing some great results (i.e. very bounded applications for algorithms initially developed, or inspired by algorithms developed for this “AI” field).

That even caused some awkward relationships at work. My manager then was a former IBM guy that was working on the AI team at IBM. A lot of people around me disregarded some of the work that we were doing because it was based on the ideas of this “AI guy”, so it wasn’t going to work.

Fast-forward to today and AI is everywhere. And now things have changed in perspective:

  1. From the general population, the biggest fear is not that it will kill us all, but that it will get rid of all jobs. A way more sensible fear based on things that go beyond Hollywood.
  2. In the research field, there is still some reluctance to call AI, as we are not at the “AI” imaged in the 80s and 90s, the one that failed miserably. We are just on a little bit less specific “expert systems” (or really “expert systems” that can learn from different applications too, but still applied to specific applications).
  3. On the professional field, people want to say that they do AI to say that they are at the edge of R&D and they are not going to be one of those companies that are going to be replaced by other company’s AIs. Yes, just like humans are afraid that they are going to be replaced by AIs, companies are afraid of the same risk.

So I think we are at a more sensible place. I personally don’t like that people are calling now any machine learning system as AI, but maybe that just softens the expectation of where we are going, making us forget a little bit that goal of this human-like machine that can out-think us. I can live with that!

Impressions from AI NEXTCon 2018

Today I went to AI NEXTCon, partly as a recruiting effort (Sift had a table there) and partly to go to the talks and see what people are talking about. It’s a fairly small conference and today was day 2 with 3 keynote speeches and two talk breakout sessions with 3-4 talks each.

It’s not a cheap conference. I think that attendance costs around $250/day. There are two conference days and two workshop days. So who attends those conferences? It was a very mixed bunch, actually. I saw some people that were fresh out of college looking for a job or to be inspired by machine learning. I saw some people that were there to sell their services (not only the ones that had a booth). There were some software professionals that were not in the AI field but curious to get into it. And there were actual AI professionals from many companies trying to see what is going on. And I think this last class is the one that was probably the most disappointed.

Talks were around 50 minutes long, so pretty long. That actually reduced the quality of the talks, in my opinion. Presenters either tried to cover a lot of ground and gave no examples, or tried to give “real” examples and ended up wasting time talking through code that probably went way too quickly to be really understood, but still took time. But I’m not going to mention names.

The only talk that I want to highlight was a talk by Amy Unruh, from Google. In that talk, she presented Google’s new AutoML. It’s still early stages for it, but I think it is a great new direction where ML-as-a-service should be going. I give some company my data and they give me a model that internally is trained with more data than the one I gave them. Hopefully that is done through transfer learning, but maybe there are other tricks that might work in fields where there is no reliable transfer learning solution.

I don’t think Google has the right product for it yet. I think there are some knobs that need to be provided for customers to do things like balancing errors between classes and other things like that, but it does have some good features:

  • Automatic separation of test set and visualization of quality in a lot of different useful ways
  • Support for asking Google to come up with people to manually label the data (apparently this is going to be staffed by Google employees/internal contractors)

The reason why I think it’s the future of ML-as-a-service is because this is where the value really is and scales. The pre-trained models are nice, but they are always hard to use in real life. The classes are either too granular, granular in the wrong places, or just not granular enough. Also it has sometimes puzzling errors (the example in the presentation above had actually a label that repeated twice with different scores – unfortunately it’s covered in the PDF version of the presentation). So you probably want to focus on the labels you care about for your application and bias training there.

If companies can provide you with a way to get the labels you want without having to spend the money to acquire the amount of data necessary to train a strong model, that’s the most sticky feature that you can provide people, and still give them what they want. I hope to see more on it soon. Image classification is an “easy” field for it (the problem is pretty well-defined and the inputs are consistent across domains, making it easier to get transfer learning to work). I will cheer more when companies start providing solutions for other fields, like many NLP tasks. Certainly something to keep an eye out for. Great start, Google!

The economics of personal data on the Internet

I was reading an article on The Economist:

Should internet firms pay for the data users currently give away?

It is an analysis of a paper “Should We Treat Data as Labor? Moving Beyond ‘Free'”
(although the article never mentions where it’s from).

While I have not read the original paper yet, here are my thoughts on the subject, that I actually even posted as comment on the article. Note that this is a reply from somebody saying that we are being paid for this data already because we are getting things for free that in the past we used to pay for (like GPS).

I completely agree that we are already implicitly paying for those services by providing our data. And that should be highlighted in the paper (it might be, but there were no references to the actual paper in the article).

I think the component that would make things more interesting is if we can make that explicit and then allow people to opt out of it (and then have to pay for the service). Interesting, but I don’t think it would be helpful at the end of the day. Data gathering from users is unbalanced. Some users give a lot of data because they use the service a lot, or have a device that is always collecting the data. And some users don’t give much data at all and don’t have the opportunity to do so. That means that the internet will become more expensive for some users, which might end up not being able to afford access and all the negative things that that entails.

I think that are many other dimensions that might be more interesting to investigate in a legal spectrum (some of them technically challenging): data portability (whatever I give to Google, I can ask them to export to GoogleNext so that they can be competitive and provide me personalized information), data sharing visibility (who gets that data and what are people outside the company allowed to query on), and data security (guarantees on what is available on a person’s raw data). That would help foster more competition while keeping data collected transparent and safe and that’s better than trying to just go the route of adding more complexity to our interactions with the internet.

Economics is hard and I don’t really have enough background to say really that the recommendation from the article is a bad one, but it feels like we are really attacking the problem the wrong direction.

At some point in the past I was asked to join a team that was handling ads on the Amazon website. My first reaction was that I was fundamentally opposed to the concept because we are muddling the relationship with customers by effectively giving something that we own from the customer (their page view) and giving a piece of it to a different company to profit from. But the reasoning was that this allowed Amazon to monetize those page views more effectively which, in turn, would allow for lower prices on the Amazon website.

That made me think about whether you can get the same effective result without the feeling that we are cheating customers to show them not something that they have asked for (and unbiased view that only contains the things that they are searching/browsing on the website). And I wasn’t able to really come up with anything better.

No, I didn’t end up going to the ads team. I decided to join a different initiative within Kindle and my path never crossed again with an opportunity doing ads, so I didn’t have to think about it anymore. Correct monetization of things is very complex, so we have sometimes to accept sub-optimal experiences in order for things to move on and, in aggregate, for you to have a better experience.

6 months away from Amazon… How does it feel?

As I’ve mentioned before, after 12.5 years at Amazon, I decided to move on and do something different. You may ask first why I left Amazon. I’ll get to that some other time. I want to first start with how is life outside Amazon. More specifically, how is life on a small mid-aged startup (6 years, past Series C) where the headquarters is in a different city where I’m working, and most people in the company have been doing software for less time than I have. That’s Sift Science, if you don’t quite know what I’m talking about.

TL; DR: it’s great! But this answer has quite a few dimensions to it:

  1. People: Sift is made of a lot of genuine people, open to feedback and learning. Amazon had a lot of those too, but for the most part Amazon’s culture focused the attention on people that were aggressive owners: they cared a lot about what they were trying to deliver and that sometimes created a little bit or us vs. them behavior.
  2. Visibility: Sift’s business is “simpler” and way more transparent to all employees than Amazon’s. Part of it is some level of maturity of the processes and staffing that Amazon has that led to a more complicated interaction between the parts of the business. And part of it is also the size of the product and the required siloing of the decision process. But, for the most part, there is some institutional distrust of people having too much information and that information potentially leaking to competitors. Back to the “people” aspect above, there is no distrust anywhere, so information flows everywhere (sometimes a little too freely, as I can see all closed deals, and ones that fell through).
  3. Software Maturity: that’s where probably there is a lot more to do at Sift than there was at Amazon. That’s expected for a fairly new company with reasonably young software developers. While the overall landscape of knowledge around availability, scalability and development best practices in the software industry has improved tremendously in the last decade, there are some components of the way software is being written that remind me of Amazon 12 years ago: monolithic structures focused on standardization and code reuse with the cost of complexity and unexpected dependencies (e.g. a table around metrics was broken and that prevented work on a job that only did a backfill for ip-to-geo mapping data).
  4. Problem space: in this one it’s a little hard to compare. At Amazon I was either working for projects that affected other teams (e.g. catalog projects to support category launches, or website feature launches), or things that directly affected customers (Amazon Go). At Sift I’m helping other companies scale their operations by externalizing the concerns about fraud detection. While we have the ability to have very close contact with those customers, in the end the impact to their business is only something we can guess or approximate. So measuring impact can be a little harder. Qualitatively, though, it’s a very important problem space, probably more important than any project that I’ve ever worked with at Amazon (unless Amazon Go actually takes off and causes a change in the way physical retail works – no signs of that yet). Getting feedback from some of our customers saying that they were only able to expand to different geographies because they trusted Sift to be able to detect fraud they were not experienced with and keep their focus on the actual “positive side” or their business is invigorating and not a rare feedback to receive.

I’m not going to deny that Amazon is an amazing company. It is honed with building processes and investments necessary to deliver things and learn from it. If that’s what you like to do and you are ready to sell your soul to get things done, I strongly recommend Amazon. Yes, it’s a large company, which means that there are pockets of everything, but in general it’s a relentless environment focused on business optimization and delivery.

Sift is not that. It is passionate, quirky, but in general sees problems in a more software-centric route. There is no strong business and process culture, but a culture of looking at the software that we’ve developed and the models that we’ve created, and making them better. That sometimes means trying some things that don’t necessarily take you anywhere, but there is no feeling of loss when that happens. We just try something else.

I don’t think I can right now claim is better or worse. It just opened my eyes to doing things differently. Internally I’m still struggling with the adjustment to it, but I can’t deny that we have very happy customers at the end of the day, so it’s certainly not the wrong way to do anything.

Maybe I’ll checkpoint my thought process again in another 6 months!