Ming the Mechanic
The NewsLog of Flemming Funch

Friday, August 12, 2005day link 

 Your blog when you're not there
It is sort of weird with a blog. It stays there even when you're not paying attention. I was busy the last few weeks, and didn't post anything, and didn't even check my comments. But still, about a thousand people are looking at it every day. OK, that's probably to a large degree because Google serves us a lot of my old postings, so it obviously isn't all folks who deliberately go and see what ming.tv has on today. But it is still a little strange, to have a lot of visitors when one is not there.

And that's where the organization of a blog is maybe not completely appropriate. You know, it shows your latest posts first on the front page. So, is it like I've been saying the same thing every day for the last month? Do my last couple of posts suddenly gain an unintended higher importance? Are thousands of people coming by, thinking, "Why did he post exactly that, and then nothing more?" I don't know.

On the other hands, there are aspects of how blogs are accessed that make it no big deal that you're not there for a while. A lot of people read blogs through blog aggregators. So, they don't lose any sleep over the fact that you haven't posted much. And the moment you post something again, they'll notice. They don't have to go checking everything day, wondering.

Likewise, this blog is part of a blog community in the New Civilization Network. Meaning, that a bunch of members watch the blogs there through a simple blog aggregator in the member area. So, they notice too right away when you post something again, and don't lose much sleep either, if I don't do it for a while.

But, still, I personally lose a bit a sleep about not blogging. A feeling like one is letting people down. That a lot of people go in vain and look for your postings, and there's nothing there. In reality, it is probably much fewer people than it feels like, but, well, it is just a feeling, not necessarily a fact.

Likewise, I'd also always have some mental obstacles to blogging again, if I haven't done it for a while. I've lost the thread, for one thing. And then I'm wondering if somehow my first posts would be particularly scrutinized. Why does he show up after a month and then post THAT? Well, the feeling goes away quickly, and I don't really worry about what anybody thinks. But the starting and stopping is a little difficult.
[ | 2005-08-12 23:00 | 2 comments | PermaLink ]  More >

 Google doesn't like me any more
picture Well, Google likes some things I do A LOT. This blog has PageRank 7, which is fabulous. But another site which I'm more concerned about is my Opentopia site. That's a site that both is intended as a service, but also to be an income producing activity. It has an assortment of openly available collections of data, like Wikipedia's encyclopedia and the Open Directory. And some more unique pieces, like the gallery of Web Cams found in Google.

Recently the site had started to look rather promising. In terms of money, that means that a lot of people come by and click on the ads. The last two months to the tune of a little more than $30 per day, or $1000 per month. That is not huge, but it is big enough to imagine I could make a living from it, if I made it better and more people came by. And the ads that the money comes from are all from Google AdSense, as that just happens to be what works best, and what most people are comfortable with.

Lots of sites link to some part of Opentopia. 3-4000. So, people are coming in from those. But the majority of people come from Google itself, from having searched on one thing or another, which exists on the site. Google had indexed a great deal of the site, so there were many entries, and the main pages got a good PageRank as well.

But, suddenly, on the first of August, Opentopia disappeared without a trace on Google. Well, not entirely without a trace. I invented the word, and it didn't exist at all a few months ago. And, today, 49,200 webpages mention the word. But, what disappeared was everything at all in Google's index that is for the Opentopia site itself. If you ask for any pages on the site, you get:
Your search - site:opentopia.com - did not match any documents.

So, zippo, zilch, nothing. Not even the home page. The site doesn't exist as far as Google is concerned. Never heard of it. No pages there. No search results. No traffic.

And, instantly, my traffic dropped, as did my Google AdSense income, to around $5 per day.

Now, normally Google is my friend. I think Google is a great company. But if they basically own the majority of the web, it is also a cause for alarm. Getting dropped from Google is a bit like having your ID card revoked by the government. You don't exist. Google entries and Google PageRank is to a large degree a currency. Something you invest in and use and spend. But your account might be emptied over night, and there's no bank teller you can go and talk to.

See, Google's operation is so huge that there isn't exactly anybody home to talk to about this kind of thing. They of course can't answer everybody's personal questions about 10 billion webpages. But how about MY website? Last month it was in the top 30,000 sites on the net in terms of traffic. That's not super-elite, but it does make me somebody a little bit. But that doesn't really make much difference.

If one has an issue with Google, there is a support form one fills in. It doesn't really matter what one fills in - one gets an automatic reply back, which refers to their FAQs, explaining the basic rules for how one needs to behave if one wants to be listed in Google. I've read all of that many times, and I think I'm following all the rules. But one of Google's algorithms must think otherwise. Mind you, one doesn't get any kind of indication about what exactly might have gotten one's site banned. Anyway, the next step is that one then writes to them again, pleading for one's case, hoping that some real person might answer, and then look at it. That might or might not happen. Depends a bit on what one writes. The best advice seems to be to write a brief message which explains that maybe one might accidentally have done something bad in the past, and one has taken steps to clean that up, and would they please, please look at the site again.

The problem is that I don't know exactly what I might have done wrong. Quite possibly nothing. But there's a lot of pages on Opentopia. More than 1 million. So it is entirely possible that Google's spider has concluded that it just goes on forever, and that it is some kind of trick, or a site filled with random junk, to attract search engine traffic.

I have indeed noticed more and more sites like that recently. Sites that include random excerpts from random other webpages. Obviously generated by some program, and obviously to get listed in search engines. And some of them have probably succeeded well. So my guess is that Google has changed something, to try to clamp down on sites with large numbers of phoney pages.

And, well, I have a lot of pages that aren't terribly original. Copies of Wikipedia and DMOZ. None of those folks have anything against other sites mirroring their content, and, for that matter, they make it relatively easy to do, by providing regular database dumps. But how does Google's spider know the difference between a mirror of Open Directory and a random content generator? I don't know. That's probably not an easy problem to solve.

One thing that might make a difference is Google SiteMaps. Essentially one generates a map of one's site and tells Google about it, and it might help their spider do a better job at indexing it. I haven't used that feature before. I'm thinking it might be helpful if they know exactly how many pages there are, so they don't think they're just beeing tricked by some auto-generated content.

Well, I don't really know. I thought I was pretty knowledgable about this kind of thing. But now I'm an outcast, pressing my nose against the window, trying to get a glimpse of what everybody else is doing inside in the warm Google livingroom.

Well, luckily I can write about it here on my Big Ass PageRank 7 WebLog. But it still isn't fair.
[ | 2005-08-12 23:48 | 16 comments | PermaLink ]  More >

Main Page: ming.tv