I have a client who has over 100+ domains. Some previous tech person had signed him up for DNS hosting by DNSMadeEasy, which is an unnecessary service if you use Cloudflare. Additionally, all those domains could be transferred out of the original registrar to Cloudflare, saving my client at least 50% costs on renewal since Cloudflare only charges wholesale prices.
What I wanted to do was to consolidate all of his profiles into Cloudflare for security, cost, and ease of management.
The first step is to create new zones (previously called sites) in Cloudflare for my client. You could do this manually, or you could take advantage of some of their command line tools to automate this process. There’s an article about this at https://developers.cloudflare.com/support/account-management-billing/account-management/adding-multiple-sites-to-cloudflare-via-automation/ however I found it lacking in some important details. First importantly is that you need to create an API token for the original owners account. So if you’re attempting to do this from your own collaborator account, you’ll have to logout and login as the client owner.
Go to the account profile > api tokens > and view the Global API key. I know the article doesn’t explain this, and other articles want you to use a token, but the Global API key is what you’re after.
Open your terminal application (e.g. Terminal, or Terminal.app) and set your API key & email:
$ export CF_[email protected]
$ export CF_API_KEY=abc123def456ghi789Then, we’ll write a simple for-loop that takes each domain name and creates a new zone for it in the account
for domain in $(cat domains.txt); do \ curl -X POST -H "X-Auth-Key: $CF_API_KEY" -H "X-Auth-Email: $CF_API_EMAIL" \ -H "Content-Type: application/json" \ "https://api.cloudflare.com/client/v4/zones" \ --data '{"account": {"id": "id_of_that_account"}, "name":"'$domain'","jump_start":true}'; done or you can use the FlareCTL tool, which is my preference.
for domain in $(cat domains.txt); do flarectl zone create --zone=$domain --jumpstart=true; doneUnfortunately it seems like there is a limit to the amount of zones you can have it one account through this method, as I was getting the error message of exceeded available rate limit retries. When attempting to add a zone manually, I got the error message of “You have exceeded the limit for adding zones. Please activate some zones.” Looking for answers in the Cloudflare community, I found this message from the admins:
We have limits on the number of pending zones allowed at any one time. If you’ve added dozens/hundreds/thousands of domains you will need to verify/validate pending domains before we’ll allow more to be added/increase the number of allowed pending zones.
Sigh, so deal with that first, then continue on….
As of the date of this article, ‘jump_start‘ doesn’t work, so I’ve got a bash script that can help with this.
You’ll need to have jq installed,
you can follow the instructions below based on your operating system:
macOS:
- If you have Homebrew installed, run the following command in your terminal:
brew install jq - If you have MacPorts installed, run the following command in your terminal:
sudo port install jq
Linux:
- Debian/Ubuntu: Run the following command in your terminal:arduino
sudo apt-get install jq - CentOS/Fedora:
sudo yum install jq - Arch Linux/Manjaro:
sudo pacman -Sy jq
Windows:
- You can download the
jqbinary for Windows from the official website: https://stedolan.github.io/jq/download/- Choose the appropriate version based on your system architecture (32-bit or 64-bit).
- Extract the downloaded archive and add the
jqexecutable to your system’s PATH.
After successfully installing jq, you should be able to use it in the bash script provided earlier to parse JSON responses and extract data. Now you’re ready to run a bash script.
Here’s a script that reads the domains from domains.txt.
The script retrieves the zone ID for each domain using a GET request to the Cloudflare API. It then performs a POST request to trigger the DNS scan for each domain by using the respective zone ID.
Make sure you have the jq command-line tool installed as it is used to parse the JSON response and extract the zone ID.
You can save the script to a file, such as bulk_dns_scan.sh, and run it in a terminal using bash bulk_dns_scan.sh.
bashCopy code#!/bin/bash
# Cloudflare API credentials
AUTH_EMAIL="YOUR_EMAIL"
AUTH_KEY="YOUR_API_KEY"
# Read domains from domains.txt
DOMAINS=($(cat domains.txt))
# Loop through the domains
for DOMAIN in "${DOMAINS[@]}"
do
# Retrieve zone ID for the domain
ZONE_ID=$(curl -s -X GET \
-H "Content-Type: application/json" \
-H "X-Auth-Email: $AUTH_EMAIL" \
-H "X-Auth-Key: $AUTH_KEY" \
"https://api.cloudflare.com/client/v4/zones?name=$DOMAIN" | jq -r '.result[0].id')
# Trigger DNS scan for the domain
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Email: $AUTH_EMAIL" \
-H "X-Auth-Key: $AUTH_KEY" \
"https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/scan"
echo "Triggered DNS scan for $DOMAIN"
done
In this updated script, the DOMAINS array is populated by reading the lines from domains.txt using the cat command. Each line in the file will be treated as a separate domain.
Make sure to replace YOUR_EMAIL and YOUR_API_KEY with your Cloudflare API credentials. Additionally, ensure that domains.txt is in the same directory as the script, or provide the appropriate path if it’s located elsewhere.
In case you’ve already added a zone, and DNS entries for any of those domains, it’ll recognize those and skip through it.
Now you’re ready to run the bash script. Type this in your terminal:
bash domains-bash.shYou should get a result something like this in the terminal:
Gray@Grays-MacMini public % bash domains-bash.sh
{"result":{"recs_added":7,"recs_added_by_type":{"A":3,"CNAME":1,"MX":2,"TXT":1},"total_records_parsed":7},"success":true,"errors":[],"messages":[]}Triggered DNS scan for domain1.com
{"result":{"recs_added":13,"recs_added_by_type":{"A":6,"AAAA":6,"TXT":1},"total_records_parsed":15},"success":true,"errors":[],"messages":[{"code":81057,"message":"domainexists.com: Record already exists."},{"code":81057,"message":"m.domainexists.com: Record already exists."}]}Triggered DNS scan for domainexists.com
{"result":{"recs_added":4,"recs_added_by_type":{"A":3,"TXT":1},"total_records_parsed":4},"success":true,"errors":[],"messages":[]}Triggered DNS scan for domaintwo.com
{"result":{"recs_added":4,"recs_added_by_type":{"A":3,"TXT":1},"total_records_parsed":4},"success":true,"errors":[],"messages":[]}Triggered DNS scan for anotherdomain.com
{"result":{"recs_added":9,"recs_added_by_type":{"A":8,"TXT":1},"total_records_parsed":9},"success":true,"errors":[],"messages":[]}Triggered DNS scan for moredomainname.comIf you have a lot of domains, this is going to take a while, so go take a break and come back after a while. If you take too long of a break, you’ll need to re-import your CF email and API key before proceeding.
The one downside to this is that this is only going to pull in DNS entries for the most common type of records, so if you’ve got subdomains on those domains, this might skip those :(
