Dev.to · 8 min read

Launch week: what to submit, what the logs should show, and how to tell a real inquiry from noise

Launch week: what to submit, what the logs should show, and how to tell a real inquiry from noise

The first week after launch is not about ranking. It is about confirming three things: crawlers can reach the site, they are actually fetching it, and the contact path works end to end. Everything else can wait until there is data. Day 0, before you tell anyone: four things that block indexing silently A coming-soon or maintenance mode that is still switched on. A noindex left over from staging. There are two places: a meta tag in the head, and an X-Robots-Tag response header. Check both. Open the page source and search for noindex, then request the page headers with curl -I. The header is the one people forget. HTTP auth or a site-wide password still enabled. The same page reachable on more than one hostname (with and without www, or both protocols) with no redirect between them. Pick one, 301 the other. One more, before launch rather than after: put the measurement tag on the site. Analytics installed on day 3 means day 1 and 2 have no data at all. Test all of this from a phone on mobile data, not from the office wifi, and not while logged in as an administrator. Logged-in views hide caching and coming-soon behaviour. Day 1: verification and submission Add a domain property, verified by a DNS TXT record. It covers every subdomain and both protocols in one property, which matters when you are not yet sure whether the canonical host is www or not. Add a URL-prefix property as well if you want per-protocol reporting. It is a second view of the same data, verified separately. Submit the sitemap once. Before submitting, open the sitemap in a browser. It has to load, return 200, be valid XML, and contain only canonical URLs that themselves return 200. No redirects, no 404s, no noindexed pages inside it. A sitemap full of redirects teaches the crawler to distrust the whole file. Then URL inspection on a handful of pages, not hundreds: home page, one category or service page, one product page, the contact page. "Request indexing" is a queue entry, not a switch. There is a daily quota and no way to buy more. Bing: import from Search Console or verify directly, then submit the sitemap there too. Ten minutes, and it feeds the results that are not Google. What not to do in week one: resubmitting the sitemap every morning, requesting indexing for every URL, or deleting and recreating the property because nothing shows on day 2. Day 2-4: crawl your own site before the crawlers do Check that every key URL returns the status you expect. One line per URL, status code first: while read -r u; do printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' -L "$u")" "$u" done < urls.txt urls.txt is one URL per line in a plain text file: home, top landing pages, contact, the sitemap, robots.txt. Then check, on each page: the canonical tag points at that page itself, not at the home page and not at the other hostname internal links use the canonical hostname; one mixed link is enough to create a duplicate the main text is in the HTML source, not injected by JavaScript. View source, not the rendered DOM. Fix these now. Crawlers come back, but they come back on their own schedule, not yours. Day 3-7: read the access log This is the step people skip, and it is the place that shows what actually happened rather than what you submitted. Where the file is. On a managed host, the control panel, usually under access logs or raw logs, rotated daily. On your own server, /var/log/nginx/access.log or /var/log/apache2/access.log. Behind a CDN, the origin log shows the CDN's IP addresses, so you need the CDN's own log or an origin header that carries the real client IP. Download a couple of days and work on the copy. # Who is hitting the site at all awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20 # How many requests claim to be a crawler grep -Eic 'googlebot|bingbot' access.log # Which paths the crawler actually fetched, busiest first grep -Ei 'googlebot' access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20 # What we served it grep -Ei 'googlebot' access.log | awk '{print $9}' | sort | uniq -c | sort -rn The field numbers ($1, $7, $9) assume the combined log format: client IP, identity, user, timestamp, request, status, bytes, referrer, user agent. Check the first line of the file and the server config before trusting them. A user agent proves nothing. Anyone can send the word Googlebot. Verify the IP by reverse lookup, then forward-resolve the name it returns and confirm it comes back to the same IP: host 66.249.66.1 # the returned name should sit under googlebot.com or google.com host If either lookup fails, or the forward lookup does not return the original IP, that hit was somebody else. Usually a scraper or a monitoring service. What real crawling looks like: more than one hit, spread over several days; requests for HTML pages rather than only images and CSS; 200 responses; and a spread of URLs that includes internal pages, not just the home page. What it does not look like: a single hit on the home page, a run of hits on the login path, or hits only on assets. Those are fetchers and scanners, not indexing. The self-inflicted block. If you switched on a firewall bot rule, a rate limit, or a security feature that blocks suspicious traffic during launch week, check it before concluding anything about the index. The log will show 403s or 429s served to a crawler. That is your own rule, not a search engine decision. Inquiries: prove the path works end to end Submit the form yourself on launch day, from a network that is not yours, using a mailbox you control. Confirm four separate things: the thank-you message appears the notification email arrives the entry is stored somewhere you can query the sender address passes SPF, DKIM and DMARC for your domain The email is the fragile part. It can fail while the form still looks fine to the visitor. Check the spam folder, then check the host's mail log for the message ID. If the mail path turns out to be unreliable, at least store every submission so nothing is lost. Server-side proof that a submission reached you, even when the email did not: grep 'POST ' access.log | awk '{print $7, $9}' | sort | uniq -c | sort -rn | head A 200 on the form handler means the data arrived somewhere. A 500 means the handler is failing and the visitor saw an error or a blank page. A 429 means your own rate limit ate a real inquiry. Separating noise from a real inquiry. Noise usually looks like this: the same IP hitting the handler several times in a few minutes, fields that are empty or obviously generated, a message body that is only a link, a referrer from a directory site. A real one usually has a company name you can look up, a reply-to address that matches the domain in the message, and a specific question about a specific thing. That is a set of heuristics, not a rule. Read the message. The week-one sheet One place, filled in daily. Five minutes a day. What Where Normal in week one Pages known / indexed Search Console pages report a small number, growing slowly Crawl requests access log, filtered by verified crawler IPs a handful per day, spread across URLs 4xx / 5xx on canonical URLs access log close to zero Form POSTs access log equals the number of inquiries you received Inquiries inbox or a shared sheet any number, including zero Top referrers analytics mostly direct and your own links early on The form POST count and the inquiry count should match. If they do not, you have a delivery problem, not a traffic problem. What not to conclude in week one Zero indexed pages on day 3 is not a penalty. Zero inquiries from fifteen visitors is not a conversion problem. It is a traffic problem, and it is too early to see either. Do not rewrite the site on five days of data. Do not resubmit anything twice. Once is the instruction; the second time is noise in your own process. The whole first week is a verification exercise: is the door open, is anyone walking through it, and does the contact path actually deliver. Everything after that is content, which runs on a much slower clock.

This is a summary aggregated from Dev.to. Read the complete article on the original site:

Read full article at Dev.to

More Cybersecurity News