Full patch 20170116195834-c9899-b886e344466c0f5f8424702fde2aad9405734613

Mon Jan 16 19:58:34 UTC 2017  Alexander Markov <apsheronets@gmail.com>
 * rss2tg_crawler

en/rss2tg_crawler
@@ -0,0 +1,20 @@
h1. rss2tg crawler

h2. I got a lot of requests from @139.162.183.118@. What's that?

I run "rss2telegram bot":http://komar.bitcheese.net/en/rss2telegram. It designed to update RSS feeds like every minute or two:

!http://munin.bullshit.agency/localdomain/localhost.localdomain/rss2tg_update_interval-day.png!:http://munin.bullshit.agency/localdomain/localhost.localdomain/rss2tg_update_interval.html

For example, you could subscribe to security updates on @https://www.exploit-db.com/rss.xml@ and receive them really, really quick via telegram messenger. It's free to use.

h2. Should I block it?

Please don't.

1 request per minute is 0.016 RPS. It's not a heavy load or DoS attack. 1000 RPS is heavy load.

This bot have @304 Not Modified@ support and sends @If-Modified-Since@ and @If-None-Match@ headers. I'll be very happy if you implement HTTP cache logic for you web application. I got a lot of traffic too:

!http://munin.bullshit.agency/localdomain/localhost.localdomain/if_eth0-month.png!:http://munin.bullshit.agency/localdomain/localhost.localdomain/if_eth0.html