Spark in me - Internet, data science, math, deep learning, philo

snakers4 @ telegram, 1298 members, 1458 posts since 2016

All this - lost like tears in rain.

Internet, data science, math, deep learning, philosophy. No bs.

Our website
Our chat
DS courses review

snakers4 (Alexander), May 19, 14:32

A thorough and short guide to Matplotlib API

A bit of history, small look under the hood and logical explanation of how to use it best:


Python Plotting With Matplotlib (Guide) – Real Python

This article is a beginner-to-intermediate-level walkthrough on Python and matplotlib that mixes theory with example.

snakers4 (Alexander), May 18, 17:47

A very cool sci-fi story about ethics, game theory and philosophy


Stupid Questions, 2nd half of December

... (Read More)

snakers4 (Alexander), May 18, 06:59

Using ncdu with exclude

A really good extension of standard du

sudo ncdu --exclude /exclude_folder /

Useful when something is mounted in /media or /mnt


snakers4 (spark_comment_bot), May 17, 09:55

Playing with 3D interactive scatter plots

Turns out you can do this using ipython widgets + ipyvolume.

Best example:

- Playing with particle data (

All of this looks kind of wobbly / new and a bit useless, but it works, is free and fast.

I was also trying to assing each point a colour like here

But in the end a much more simple approach just worked

fig = ipv.figure()

N = len(hits.volume_id.unique())

cmap ="tab20", N)

colors = cmap(np.linspace(0, 1.0, N))

colors = ["#%02x%02x%02x" % tuple([int(k*255) for k in matplotlib.colors.to_rgb(color)[:3]]) for color in colors]

for i in range(0,N):

hits_v = hits[hits.volume_id == list(hits.volume_id.unique())[i]]

scatter = ipv.scatter(hits_v.x, hits_v.y, hits_v.z, marker="diamond", size=0.1, color=colors[i])

Like this post or have something to say => tell us more in the comments or donate!

snakers4 (Alexander), May 16, 13:21

A very cool, fast and simple way to make presentations

The Easiest Way to Make Presentations! (Pandoc + Markdown)
Support the channel!: Give me money: Ask a question: [email protected] Get my configs: gi...

snakers4 (Alexander), May 15, 17:11

AI Learns Painterly Harmonization | Two Minute Papers #249
The paper "Deep Painterly Harmonization" and its source code is available here:

snakers4 (Alexander), May 15, 06:18

Internet / tech

- Google I/O news

- MS to give custom voice option to its apps -

- Katzenberg (former Disney executive) raises US$800m to make YouTube like short series - => Internet + Video is a commodity now?

- _Reportedly_ Lyft has 35% market share in the USA

- Google becoming evil and doing military contracts - - wtf?

- Apple autonomous drive fleet is _repotedly_ now at 55


The 10 biggest announcements from Google I/O 2018

Here’s the most important news to know.

snakers4 (Alexander), May 15, 05:04

A great presentation about current state of particle tracking + ML

Also Kaggle failed to share this for some reason

Key problem - current algorithm - Kalman filter faces time constaints


snakers4 (Alexander), May 13, 12:25

How to show links?

Full if short, hide if long – 27

👍👍👍👍👍👍👍 56%

Hide them behind markup – 11

👍👍👍 23%

Post full links – 9

👍👍 19%

Shorten them – 1

▫️ 2%

👥 48 people voted so far.

snakers4 (spark_comment_bot), May 13, 11:25

2018 DS/ML digest 10


(0) Some moonshots by Google in working with electronic health records

(1) Google duplex - a narrow domain bot that makes calls for you

(2) Nature wants to make its ML journal ... paid

(3) Standford DawnBench - training Imagenet encoders as quickly and cheaply as possible

(4) Facebook achieves 85% on Imagenet by training on 1bn images in 336 GPUs in a week

(5) Learning the models of the surrounding world based on a DOOM like game

Practice / libraries / code

(0) A smarter and new way to ensemble CNNs

- Traditional approach - ensemble CNNS with different architecture - and just vote / average / apply linear regression on top

- Newer approach - use Cyclic Learning rate

- Even newer approach - model snapshot ensembling

- Stochastic Weight Averaging

-- store running average of the models

-- train one model with CLR

-- at the end of each lr update (or epoch) - do a running average of the models with some weights

-- the gist of the method is located on this line

-- I do understand why the update bnorm params, but I do not understand why it cannot be done just running 1 train epoch

- Papers on CNN ensembling 1 2 3

(1) (RU) Small amount of technocal details, but face-detection + face hashing works in retail (+human operator) given an HD camera

(2) (RU) Pose estimation

(3) Numpy autograd

"New" papers worth mentioning

(0) SqueezeNext

- Module comparsion

- Key changes

(i) more aggressive channel reduction by incorporating a two-stage squeeze module

(ii separable 3 × 3 convolutions

(iii) element-wise addition skip co

nection similar to ResNet

- Performance

(1) GANs to generate full-body anime characters in different poses


(0) (does not work in Firefox) Visualizing encoder-decoder networks for translation




Like this post or have something to say => tell us more in the comments or donate!

Deep Learning for Electronic Health Records

Posted by Alvin Rajkomar MD, Research Scientist and Eyal Oren PhD, Product Manager, Google AI When patients get admitted to a hospital, th...

snakers4 (Alexander), May 13, 07:26

Presentation from a winning solution in DS Bowl 2018


snakers4 (spark_comment_bot), May 12, 15:40

Google Duplex

In a nutshell - a combination of ML (RNN + speech recognition + WaveNet + Tacotron) that can call a human and pretend to be a human. Only works for narrow specific domains (call to a restaurant).


(0) Blog post

(1) MKBHD video

Usually I include such links into digests, but this time it looks like insanity:

(0) It looks heavily doctored, but believable

(1) All the components are kind of known to be at least 90-95% as good as presented

(2) Once again - it is very domain focused

One of the key research insights was to constrain Duplex to closed domains, which are narrow enough to explore extensively.

Duplex can only carry out natural conversations after being deeply trained in such domains.

It cannot carry out general conversations.

Like this post or have something to say => tell us more in the comments or donate!

Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone

Posted by Yaniv Leviathan, Principal Engineer and Yossi Matias, Vice President, Engineering, Google A long-standing goal of human-comput...

snakers4 (spark_comment_bot), May 12, 07:25

Running mean programming pattern in PyTorch

Sometimes you just need to apply exponential weighting:

(0) When tracking some metric

(1) When weighting a loss

(2) When applying something inspired by Adam

I used to do it in a quite an ugly way:

(0) Feed a list => calculate averages

(1) Do the same, but using a separate class

Found out from my colleague that it can be done in PyTorch using torch.nn.register_buffer

Very cool

Like this post or have something to say => tell us more in the comments or donate!

snakers4 (spark_comment_bot), May 11, 16:36

Testing comments on

Some time ago we saw how some channels (like HackerNews) format their posts in Telegram:

(0) The post itself contains a button with a link to a comment section of their website

(1) The comment counter is updated mostly in line with real comment count on the website

(2) Sometimes there are additional buttons

This can be easily done using using third-party bots, but in our case given our custom-built blog, there was only one way:

(0) Create a post on the blog with an embedded Telegram snippet, hide it from all of the feeds

(1) Create a simple bot for posting messages (like this one)

(2) Create a simple bot connected to disqus API and updating the comment counter

(3) Profit!

For some topics the community opinion / knowledge / experience may be worth much more than a single post (like GANs or proxy setup).

So, if you like this feature - please tell use what you think in the comments!

Exploring the limits of unsupervised Machine Learning in Computer Vision

In this article I share my experience with GANs, progressive growing of GANs, image clustering and unsupervised learning Статьи автора - Блог -

snakers4 (Alexander), May 10, 18:43

This AI Reproduces Human Perception | Two Minute Papers #248
Our Patreon page: One-time payment links are available below. Thank you very much for your generous support! PayPal: ...

snakers4 (Alexander), May 10, 13:41

Forwarded from Hacker News:

Ubuntu 18.04: Unity is gone, Gnome is back, and Ubuntu has never been better (Score: 100+)



Ubuntu 18.04: Unity is gone, GNOME is back—and Ubuntu has never been better

Server users will really like 18.04, but the newest Ubuntu works great for all Linux fans.

snakers4 (Alexander), May 10, 12:03

Forwarded from

Яндекс «исправил» невысокое разрешение «интернет-версиям» 7 классических фильмов о ВОВ. Формально плёнки можно просто заново классно оцифровать, но поисковик провёл другой эксперимент. Включил нейронную сеть и с её помощью добавил деталей и убрал технический брак существовавшим не ахти оцифровкам.

Нейронная сеть «Яндекса» подняла качество старому кино о Великой Отечественной войне

Искусственный интеллект помог сделать кадры детальнее, визуально более резким, а также компенсировал некоторые технологические недостатки оцифровки // Роем в вашем Телеграме:

snakers4 (Alexander), May 10, 05:39


Interesting links about Internet

(0) Ben Evans

Russia / CIS

(0) Telegram has a new proxy setting in alpha, though no proper stand-alone solutions are published

(1) Western media now cover Telegram

Global / tech

(0) Xiaomi to file for an IPO - US$10 - US$100bn

(1) Yet another drag and drop ML that will (m?) fail - - this is so American

(2) Now all "major" apps heavily feature "stories" as main mobile format -

Yet another reason to quit all social media and just use professional apps / messaging

Add up all this bs => this is the reason normal people do not use social media for real now

(3) Tesla most shorted tech company now - xD


(0) YouTube - 1.8bn users with 1+ login

(1) WhatsApp m70bn messages per day (vs. 20bn max with SMS)



snakers4 (Alexander), May 09, 06:55

A couple of articles about the harsh reality of DS / ML jobs

In a nutshell:

- politics

- unjustified decisions

- same as everywhere



True story, especially about political decisions, fight for power, useless dashboards and data monkeys.


The most difficult thing in data science: politics

Deep learning looks difficult to you? Come back after you get to know company politics, it will feel like a breeze...

snakers4 (Alexander), May 07, 18:09 publishing​ a 2018 version of cutting edge course

Their materials are cool, but their library is questionable


snakers4 (Alexander), May 06, 16:45

press release

This AI Learns From Its Dreams | Two Minute Papers #247
The paper "World Models" is available here: Support the series and pick up cool perks on our ...

snakers4 (Alexander), May 05, 13:13

Andrew Ng book

Is being released on chapter-by-chapter basis


This book is not really technical, though - it's more or less a combination of advice how to build ML models as a business process

Interesting idea - splitting your dev set into black box and eyeball dev set ... but this can be replaced by properly using Tensorboard when training...


snakers4 (Alexander), May 04, 07:45

A decent explanation about decorators in Python


snakers4 (Alexander), May 04, 06:59

The current state of ML

(1) Do not call it AI

(2) Distinguish ML from Intelligent Infrastructure and Intelligence Augmentation

(3) Human-imitative AI is not tractable now

(4) Developments which are now being called "AI" arose mostly in the engineering fields associated with low-level pattern recognition and movement control


Artificial Intelligence — The Revolution Hasn’t Happened Yet

Artificial Intelligence (AI) is the mantra of the current era. The phrase is intoned by technologists, academicians, journalists and…

snakers4 (Alexander), May 04, 05:47

Add comment button below major posts?

Yes, definitely! – 26

👍👍👍👍👍👍👍 57%

Meh... – 14

👍👍👍👍 30%

No, why? – 5

👍 11%

Your option (PM / chat) – 1

▫️ 2%

👥 46 people voted so far.

snakers4 (Alexander), May 03, 02:38

How Computers Find Naked People in Photos - YouTube
Why isn't the internet just covered in naked people? Algorithms! However, designing them to distinguish between pornography and people in skin tone clothing ...

snakers4 (Alexander), May 01, 16:52

2018 DS/ML digest 9

Market / libraries

(0) Tensorflow + Swift - wtf -

(1) Geektimes / going international -

(2) A service for renting GPUs ... from people

- Reddit

- Link

- Looks LXC based (afaik - the only user friendly alternative to Docker)

- Cool in theory, no idea how secure this is - we can assume as secure as providing a docker container to stranger

- They did not reply me in a week

(3) A friend sent me a new list of ... new yet another PyTorch NLP libraries

-, (AllenNLP is the biggest library like this)

- I believe that such libraries are more or less useless for real tasks, but cool to know they exist

(4) New SpaceNet 4?

(5) A new super cool competition on Kaggle about particle physics?

Tutorials / basics

(0) Bias vs. Variance (RU)

(1) Yet another magic Jupyter guideline collection -

Real world ML applications

(0) Resnet + object detection (RU) - people wo helmets 90% accuracy -

(1) about using embeddings with Tabular data -

Very similar to our approach on electricity

I personally do not recommend using their library by all means

(2) Comparing Google TPU vs. V100 with ResNet50 -

- speed -

- pricing -

- but ... buying GPUs is much cheaper

(3) Other blog posts about embeddings + tabular data

- Sales prediction

- Taxi drive prediction

MLP + classification + embeddings - /

(4) Albu's solution to SpaceNet - augmentations

CNN overview

Neural network part:

Split data to 4 folds randomly but the same number of each city tiles in every fold

Use resnet34 as encoder and unet-like decoder (conv-relu-upsample-conv-relu) with skip connection from every layer of network. Loss function: 0.8*binary_cross_entropy + 0.2*(1 – dice_coeff). Optimizer – Adam with default params.

Train on image crops 512*512 with batch size 11 for 30 epoch (8 times more images in one epoch)

Train 20 epochs with lr 1e-4

Train 5 epochs with lr 2e-5

Train 5 epochs with lr 4e-6

Predict on full image with padding 22 on borders (1344*1344).

Merge folds by mean

Jobs / job market

(0) Developers by country by scraping GitHub -

- developers count vs. GDP R^2 = 84%

- developers count vs. population - R^2 = 50%


(0) Interactive tool for visualizing convolutions -


(0) Open Images v4 outsourced


- the dataset itself

- categories





swift - Swift for TensorFlow documentation repository.

snakers4 (Alexander), May 01, 07:53

Exploring GANs and unsupervised learning

Here are my findings from my hobby project about using GANs and unsupervised methods to build some decent semantic search on a large dataset of images without annotation:


Lots of cool images.


(0) Features from pre-trained Imagenet encoder => PCA => Umap => HDBSCAN work really well for image clusterization;

(1) Any siamese network / hard negative mining inspired methods just did not work - the annotation data is too coarse;

(2) GANs kind of work, but I could not achieve the boasted photo-realistic levels;


Exploring the limits of unsupervised Machine Learning in Computer Vision

In this article I share my experience with GANs, progressive growing of GANs, image clustering and unsupervised learning Статьи автора - Блог -

snakers4 (Alexander), May 01, 06:58

Showing more images in Tensorboard

TB is super cool (also in together with script, but it shows ~10 images in its image preview.

This can be fixed.

(0) Find your TB folder

import tensorboard

tensorboard.__file__In my case it shows '/opt/conda/lib/python3.6/site-packages/tensorboard/'


cd there

open backend/


Change this line

image_metadata.PLUGIN_NAME: 400,(3)

Profit - now it shows ~400 images on each view tab


Logging to tensorboard without tensorflow operations. Uses manually generated summaries instead of summary ops

snakers4 (Alexander), May 01, 06:51

Pinned post

What is this channel about?

(0) This channel is a practitioner's channel on the following topics: Internet, Data Science, Deep Learning, Python

(1) Don't get your opinion in a twist if your opinion differs. You are welcome to contact me via telegram @snakers41 and email - [email protected]

(2) No BS and ads


(0) Become a patreon 🤟 -

(1) Buy me a coffee 🤟

Give us a rating:


Our chat


More links

(0) Our website

(1) Our chat

(2) DS courses review

(3) GAN papers review

(4) SpaceNet Challenge

(5) DS Bowl 2018

(6) Data Science tag on the website

Patron Checkout | Patreon

Patreon is empowering a new generation of creators. Support and engage with artists and creators as they live out their passions!