Webmaster Support

play_circle_filled

About

Sharing useful tools and information for Managers, Developers and System administrators of web applications. As a start only Whois database is being released to help with rate control issues.
  • I Like
  • these
  • Menus

History

Rest API deployed to Production
While the Rest API is ready, there is no documentation to go with it. Hence this domain was recycled and a good time to use NPM as well as NodeJS to make a static site that is multi-device. Here boilerplate from Google, Web Starter Kit came in handy. A good time to re-learn all the HTML from 18 years ago that was mostly learned offline using Internet Explorer 3.0 error page source code as reference.
30% through Whois data collection, Rest API ready
After a few setbacks with Python module for a Cloud based CouchDB, restarted locally and as of date the IPv4 range is 30% covered. Somewhere along the line began working on Rest API and its now ready for deployment. This is where the Python microframework known as Flask was used for the API. Needs improvement in specific IP matches but otherwise works fine for collecting iterative data.
started Collecting Whois data
Half way through collecting whois data and storing in JSON format. The purpose is to collect IP data and be able to query it manually or automatically in programs. Fortunately the RDAP format is now available with all RIR, the last one being AFRINIC. This saves a lot of trouble, however every RIR has rate controls in place. As such the data collected and refreshed over time will be made available without rate controls.
  • Useless
  • Menu
  • Feature

Technology

The site is new and using a boiler template. The site will be updated as and when there is content and perhaps some small improvements along the way. For now enjoy trying to figure out which links are actually working.
  • Menus..
  • they
  • stay

Data

Databases and Data by API are being made available for anyone to use.
Rate Limits

Currently there isn't any specific rate limit. The expectation here is for the end user to be considerate. 2 queries per second is acceptable maximum for any use case. IP Blocking is in place for permanent denial of service for the most abusive IPs only. Idea is to enforce some logic instead of blindly querying the API. Use caching where you can, use "Accept-encoding: gzip" if you can.

Whois API
The Whois data converted from RDAP to a local JSON format is available and continuously updated. The basic usage is outlined below.
  • Path: /iputils/whois/
  • Endpoint: http://www.webmastersupport.com/iputils/whois
  • Method: GET
  • Arguments: IP Address / (optional: subnet mask)
The output returned is of type application/json with most of the RDAP data in JSON format or HTTP Status 404 is returned if not found. The data is not according to RDAP format. Private IP or Reserved Addresses will also be returned with data same as RDAP format. The database is still being developed so you may get missing items when they should exist. The only guarantee is that whatever data is indeed available is accurate as of the time it was collected.

Example: GET http://www.webmastersupport.com/iputils/whois/60.0.0.0

Dealing with Rate limits is easy for this one. It is best to query for the start address and skip to the end address in the CIDR block of addresses. Query for the next address. Any programming language of your choice will have basic IP and Subnet manipulation librarylike Python's netaddr

Security

Performance

Dive into Performance. Start with what this site is built on while this page is being updated.

Cloud

You can read my blog posts over here in the meanwhile old Blog