AboutSharing useful tools and information for Managers, Developers and System administrators of web applications. As a start only Whois database is being released to help with rate control issues.
Rest API deployed to ProductionWhile the Rest API is ready, there is no documentation to go with it. Hence this domain was recycled and a good time to use NPM as well as NodeJS to make a static site that is multi-device. Here boilerplate from Google, Web Starter Kit came in handy. A good time to re-learn all the HTML from 18 years ago that was mostly learned offline using Internet Explorer 3.0 error page source code as reference.
30% through Whois data collection, Rest API readyAfter a few setbacks with Python module for a Cloud based CouchDB, restarted locally and as of date the IPv4 range is 30% covered. Somewhere along the line began working on Rest API and its now ready for deployment. This is where the Python microframework known as Flask was used for the API. Needs improvement in specific IP matches but otherwise works fine for collecting iterative data.
started Collecting Whois dataHalf way through collecting whois data and storing in JSON format. The purpose is to collect IP data and be able to query it manually or automatically in programs. Fortunately the RDAP format is now available with all RIR, the last one being AFRINIC. This saves a lot of trouble, however every RIR has rate controls in place. As such the data collected and refreshed over time will be made available without rate controls.
TechnologyThe site is new and using a boiler template. The site will be updated as and when there is content and perhaps some small improvements along the way. For now enjoy trying to figure out which links are actually working.
DataDatabases and Data by API are being made available for anyone to use.
Currently there isn't any specific rate limit. The expectation here is for the end user to be considerate. 2 queries per second is acceptable maximum for any use case. IP Blocking is in place for permanent denial of service for the most abusive IPs only. Idea is to enforce some logic instead of blindly querying the API. Use caching where you can, use "Accept-encoding: gzip" if you can.
Whois APIThe Whois data converted from RDAP to a local JSON format is available and continuously updated. The basic usage is outlined below.
- Path: /iputils/whois/
- Endpoint: http://www.webmastersupport.com/iputils/whois
- Method: GET
- Arguments: IP Address / (optional: subnet mask)
Example: GET http://www.webmastersupport.com/iputils/whois/22.214.171.124
Dealing with Rate limits is easy for this one. It is best to query for the start address and skip to the end address in the CIDR block of addresses. Query for the next address. Any programming language of your choice will have basic IP and Subnet manipulation librarylike Python's netaddr