- Blogging is dead, but have we fixed anything?
Google Reader is shutting down, but most people moved on long ago.
Blogging is dead. To the extent that it lives, it is dominated by professional journalists, writers backed by major organizations, or has transformed into microblogging. The original objective of an amateur form of journalism -- long articles written and published without an organization or editor -- has become archaic.
I have been writing on this blog since 2004. At its peak, this blog had about 10k regular readers. Over a decade, I have watched blogging rise and fall.
Nowadays, my posts here on this blog often get less attention that my tweets on Twitter. 140 characters that take two minutes to spew out sometimes get more attention than an article that takes four hours of thoughtful analysis, careful reading, and tight writing.
There is nothing wrong with people moving on. Professional journalists now use blogs to air early research or analysis that will later make it into a full print article. Companies use blogs to announce changes or new features. Many use microblogging as a useful means of quick communication. That is good.
But there was something charming about so many people trying to be amateur journalists. Journalistic writing is a skill; it emphasizes clear, tight, concise writing. That so many were attempting it and practicing it had a lot of value, both in the the skills bloggers gained and sometimes candid and insightful articles produced.
I find my blogging here to be too useful to me to stop doing it. I have also embraced microblogging in its many forms. Yet I am left wondering if there is something we are all missing, something shorter than blogging and longer than tweets and different than both, that would encourage thoughtful, useful, relevant mass communication.
We are still far from ideal. A few years ago, it used to be that millions of blog and press articles flew past, some of which might pile up in an RSS reader, a few of which might get read. Now, millions of tweets, thousands of Facebook posts, and millions of articles fly past, some of which might be seen in an app, a few of which might get read. Attention is random; being seen is luck of the draw. We are far from ideal.
Attention should flow to relevant and useful writing. I should see writings that are personally relevant and useful to me. When a friend does something I want to know about, when a colleague reads an article I should read too, when a company announces a useful change to a product I use, when a well-written article important for my work is published from a reputable source, when a major event occurs in the world, those should be brought to my attention.
Blogging wasn't that, but neither is microblogging. We need to build something that focuses our attention, improves our communication, and finally solves the problems blogging and microblogging failed to solve.
- More quick links
Again, it has been too long, but here you go, what has caught my attention lately:
- "Employees who ate at cafeteria tables designed for 12 were more productive than those at tables for four, thanks to more chance conversations and larger social networks. That, along with things like companywide lunch hours and the cafes Google is so fond of, can boost individual productivity by as much as 25 percent." ()
- "Managers avoid dealing with low performers (because they believe the conversation will be difficult), and instead assign work to the employees they enjoy — i.e. high performers ... They end up 'burning out' those same high performers." ()
- "Is it really true that using someone else's invention is the actually the same thing as stealing their sheep? If I steal your sheep, you don't have them any more. If I use your idea, you still have the idea, but are less able to profit from using it. The two concepts may be cousins, but they not identical." ()
- Clever and simple idea: Attach a little flash memory and a small battery to memory chips ( )
- Another clever and simple idea: On touchscreens (like your phone), make a knuckle or nail tap like a right mouse click so it does something different (  )
- Most data visualizations would be more clear done as a simple bar chart ()
- When someone comes back to a search result page after hitting the back button, you should add more search results to the bottom of the page ()
- For the first time, more smartphone ship than dumbphones, which has big implications, especially for the developing world ( )
- You can identify people based on just four locations sampled from a mobility trace (cell towers and Wifi nearby) from their cell phone ()
- "The problem is that Apple has not been able to sustain its high margin levels" ( )
- Humor (from The Onion): Weeping Tim Cook spotted screaming for help at Steve Jobs' tombstone ()
- Amazingly arrogant executive hired from Apple didn't understand customer base or think he had to, destroyed a major retailer ( )
- Amazon moves against Google ( ) and Google moves against Amazon (   )
- Very soon, only big players -- like Amazon, Facebook, and Google -- will be able to do personalized advertising. A change to third-party cookies will kill off all startups working on personalized advertising, but major websites get an exemption. ( )
- A new compression library from Google designed for web content, can be decompressed by existing software so no changes required on the client side to use it, just need to recompress the static content on the server to save about 5% in bandwidth ()
- eBay successfully moves away from auctions. "Auctions ... are less than 10% of what we do." ()
- "At this point, unfortunately, it seems clear that the Windows 8 launch not only failed to provide a positive boost to the PC market, but appears to have slowed the market ... Radical changes to elements like the user interface and higher costs had made PCs less attractive compared with tablets and other devices." ( )
- A MacBook Pro runs Windows faster than any PC laptop (but only because PCs have so much crapware installed) ( )
- "Aereo's founders realized that [a court] ruling offered a blueprint for building [an IPTV] service that wouldn't require the permission of broadcasters. In Aereo's server rooms are row after row of tiny antennas mounted on circuit boards. When a user wants to view or record a television program, Aereo assigns him an antenna exclusively for his own use." ()
- The vast majority of people have simple taxes, so simple that the IRS could just mail you a tax return, you'd look it over to make sure everything is correct and sign it, and you'd be done. Why don't we have that? Apparently, "it's been opposed for years by the company behind the most popular consumer tax software—Intuit, maker of TurboTax." ()
- Why Redfin has been unable to undermine the absurdly high 6% commission when you sell your home ( )
- "Personal finance courses ... have no effect on financial outcomes ... [but] additional training in mathematics [does]" ()
- "Graduate school in the humanities: Just don't go" ( )
- At least so far, MOOCs (like Coursera and Udacity) seem to only work for people who are already highly motivated, which isn't the group in the most need ()
- Seems to be increasing evidence that some autoimmune diseases (including allergies) are rooted in a bored immune system incorrectly prioritizing threats. Almost a parallel with anxiety disorders, your immune system is seeing threats where none exist, incorrectly prioritizing dangers. ( )
- "Deep waters have absorbed a surprising amount of heat -- and they are doing so at an increasing rate over the last decade" ()
- "Resilience -- building systems able to survive unexpected and devastating attacks -- is the best answer we have right now." ()
- The web-based version of blackmailing people who have done something embarrassing ()
- Little known fact, the second most used web server is something called Allegro RomPager ( )
- For most people in the US, the vast majority of entertainment time is still spent watching normal, live TV ()
- Odd similarities between distributed denial of service attacks and pollution. As Ed Felten writes, misconfigured DNS servers allow massive DDoS attacks, but it's hard to get people to fix it, because "the resulting harm falls mostly on people outside the organization." ( )
- Quick links
It's been a while since I did a Quick Links post, so there's a lot to cover. Here's the latest of what has caught my attention:
- First Netflix wanted to be Blockbuster (DVDs), then a replacement for cable (streaming video), now they want to be HBO (making content).
(  )
- "For raw bandwidth, the internet will probably never beat SneakerNet"
- Data caps are "a strategy for ISPs to increase their revenue per user ... The trend is driven in large part by a woefully uncompetitive market that allows the nation's largest providers to generate enormous profits"
- "Maybe it will eventually dawn on [ISPs] that the only way to fight the scourge of cheap, fast broadband is to provide it themselves"
- "Too many companies think of their call centers as a cost to minimize ... it's a huge untapped opportunity ... [for] word-of-mouth marketing ... [and] to increase the lifetime value of the customer"
- Mary Jo Foley says, "I keep scratching my head over who Microsoft expects to buy the Surface Pro"
- "Taking the bitter pill would mean backing off the Surface idea while smoothing over the worst parts of Windows 8. Admit that being different just for the sake of being different is a losing strategy. Go back to software engineering 101. But I don't see Ballmer making that tough decision. It's just not how he rolls. Then it'll be up to the board of directors to hold him responsible when this dogmatic strategy fails."
- "Dell outsourced the management of its supply chain, and then the design of its computers themselves. Dell essentially outsourced everything inside its personal-computer business—everything except its brand— to Asus ... Then, in 2005, Asus announced the creation of its own brand of computers. In this Greek-tragedy tale, Asus had taken everything it had learned from Dell and applied it for itself."
- "The Dreamliner was supposed to become famous for its revolutionary design. Instead, it’s become an object lesson in how not to build an airplane"
- A deal protects Apple, Google, and a few others from being sued by Kodak's patents, but no one else. "Kodak patents may well be popping up in future patent troll suits in the future."
- Mark Cuban says, "Dumbass patents are crushing small businesses"
- Detailed technical discussion of the Superbowl power outage and what could have been done to prevent it
- The book "Thinking Fast and Slow" and implications for artificial intelligence
- "We understand the meaning of an object in terms of the meanings of other objects – other chunks of reality to which our brains have assigned certain characteristics. In the brain’s taxonomy, there are no discrete entries or 'files' – just associations that are more strongly or more weakly correlated with other associations ... Might 'meaning' itself simply be another word for 'association?'"
- On global warming: "There is only one thing we can do: develop renewable technologies that are substantially cheaper than coal, and give these technologies to the developing countries."
- Good summary of a Davos panel on education
- Funding at Garfield High School in Seattle is just $5,600/year/student
- Fascinating example of novel work in a field (in this case, literature) by blending it with computer science.
(  )
- Companies should stop talking about "mobile", start splitting out tablets and smartphones separately.
- People talk about tablets killing the PC, should be talking about tablets killing the e-reader
- Clever optimization idea from Google: "sending a hedging request after a 10ms delay reduces the 99.9th-percentile latency for retrieving all 1,000 values from 1,800ms to 74ms while sending just 2% more requests."
- "Any time you access Google, you probably are in a dozen or more experiments"
- What could we do in a distributed database if we could rely on all servers having exactly the right time?
- Spotify rediscovers what others found a decade ago, social recommendations don't work, that "no matter who you are, someone you don't know has found the coolest stuff."
(  )
- "Amazon sells things to people at prices that seem impossible because it actually is impossible to make money that way .... Competition is always scary, but competition against a juggernaut that seems to have permission from its shareholders to not turn any profits is really frightening."
- Amazon goes after personalized ads: "This platform lets the company retarget its users across the Web based on their browsing and purchase habits on Amazon’s owned-and-operated properties. That could be a game changer ... given Amazon's recommendation engine"
- "Consumers want more targeted and humorous ads ... 67 percent of respondents would be willing to be answer a question to make their ads more personalized and enjoyable ... Consumers understand the exchange of free content for advertising, but they want to make sure their time tradeoff of watching ads also benefits them. They found coupons, contests and links as the most positive forms of engagement."
- "Advertisements are 182 times as likely to deliver malicious content than pornography"
- Dilbert on effective mobile advertising
- The future of maps on smartphones: "It'll be like you're a local everywhere you go. You'll know your way through the back alleys and hutongs of Beijing, you'll know your way all around Paris even if you've never been before. Signs will seem to translate themselves for you. This kind of extra-smartness is coming to people."
- Shocking to see Acer bragging about Google Chromebook sales while lambasting slow Windows 8 sales
- Chromebook is the #1 selling laptop on Amazon.com right now, not Apple, not Microsoft's Windows 8.
- Marissa Mayer says, "In the future, you'll be the query"
(  )
- Recommendation algorithms work by finding things other people loved that you haven't found yet and bringing them to your attention. It's computers helping humans help humans.
- A good UX can make people very forgiving of high error rates
- Stephen Wolfram says, "If heuristics are done well, with serious computation and knowledge behind them, they actually do work, and people like them very much ... So long as everything 'just works', people never think about the heuristics, never try to deconstruct them, and never notice or get confused by the lack of ultimate consistency."
- Google discovered the optimal length of an interview loop is 4 interviews. Any more hits diminishing returns. )
- "Granting mothers five months of leave doesn't cost Google any more money."
- "Software development at Google is big and fast. The code base receives 20+ code changes per minute and 50% of the files change every month"
- Worth knowing and understanding: Android has 42% market share of computing devices, but only generates 5% of Wikipedia's traffic
- "Why the Google+ long game is brilliant"
- Snarky: "The real sign of Google Apps making a big dent in the business world will be when its own hiring managers are able to stop treating Microsoft Office as the de facto standard."
- "When everything is in flux, predicting what will be hot a year from now -- 'skating to where the puck is going to be,' to quote Steve Jobs quoting Wayne Gretzky -- becomes all but impossible. Samsung's strategy is to put a man at every spot on the ice. Be in enough places and you're bound to catch something no one was predicting -- like, for instance, the world’s bizarre love affair with phablets."
- Much lower power consumption on GPS trails on smartphones by offloading processing to the cloud
- Clever combination of GPS trails and a game: "The idea of cyclists recording ride data is nothing new ... What Strava did was turn ... [that] into a rigorously measured, database-matched, global community with the sudden ability to turn the most banal ride into a race ... Get that satisfaction without turning up at the starting line, in the rain, on a Saturday morning at 6 a.m."
- Interesting theory: "I had a small epiphany. The cyclists were hated because they are [viewed as] cheats. They are getting away with something that car drivers cannot."
- I love this idea of a bicycle frame completely covered in reflective paint
- Out of control: "The American Civil Liberties Union filed a Freedom of Information Act request with the FBI seeking details of its surveillance policy -- who it spies upon, and how, and under what circumstances. The FBI sent back two 50+ page memos in reply, each of them totally blacked out except for some information on the title page"
- On hedge funds: "The S&P 500 has now outperformed its hedge-fund rival for ten straight years, with the exception of 2008 when both fell sharply. A simple-minded investment portfolio—60% of it in shares and the rest in sovereign bonds—has delivered returns of more than 90% over the past decade, compared with a meagre 17% after fees for hedge funds (see chart). As a group, the supposed sorcerers of the financial world have returned less than inflation. Gallingly, the profits passed on to their investors are almost certainly lower than the fees creamed off by the managers themselves." )
- Appears both Vikings and Polynesians reached the Americas around 1000, well before Christopher Columbus
- The weight of glaciers during ice ages might cause an increase in volcanic eruptions
- Moderate amounts of play of first person shooters (and similar action games) improve vision, attention, and spatial skills
- Randall Munroe (author of xkcd): "I've never seen the Icarus story as a lesson about the limitations of humans. I see it as a lesson about the limitations of wax as an adhesive."
- An art project with a visible pile of pennies and a crank, that "allows anyone to work for minimum wage for as long as they like." Absolutely brilliant.
- Kids, programming, and doing more
I built Code Monster and Code Maven to get more kids interested in programming. Why is programming important?
Computers are a powerful tool. They let you do things that would be hard or impossible without them.
Trying to find a name that might be misspelled in a million names would take weeks to do by hand, but takes mere moments with a computer program. Computers can run calculations and transformations of data in seconds that would be impossible to do yourself in any amount of time. People can only keep about seven things in their mind at once; computers excel at looking at millions of pieces of data and discovering correlations in them.
Being able to fully use a computer requires programming. If you can program, you can do things others can't. You can do things faster, you can do things that otherwise would be impossible. You are more powerful.
Looking two decades out, when my kids are grown and well into their careers, I expect people who can fully use computers will have a major force multiplier. A blend of computer science and another field -- medicine, microbiology, genetics, economics, astronomy, journalism, business, almost anything -- will enable you to do things others in that field can't.
Already you can see this. Breakthroughs in genetics came from a collaboration between computer science and geneticists working to create new algorithms for massive scale approximate string matching. During the 2012 elections, Nate Silver redefined what it meant to be a journalist (and attracted huge amounts of traffic) by combining computing and large amounts of polling data in a new way. Astronomy is becoming a field of big data, computers analyzing huge amounts of data from a worldwide network of telescopes, pulling out promising patterns, then having humans look over the candidates to find new discoveries. Robotic probes and the massive data streams they produce are not only taking over space exploration, but also making inroads on sea exploration, marine biology, and climatology as well. Already, if you can program, you can do things others cannot, find things others cannot.
Over the coming years, the collaboration between computers and machine is only going to grow. Computers will do what they are good at, large scale data processing, computation, and analysis. Humans will do what they are good at, finding patterns, intuiting promising paths forward despite noise and missing data, and collaborative problem solving. Those who can fully use computers, and especially those who can program computers, will be more productive. Computers are a powerful tool for those who can wield it.
Sadly, many kids today think of programming as hard. As not fun. As not for them. The problem is particularly acute for girls, leading to the awful fact that only 14% of the computer science degrees in the US are awarded to women. So many kids not getting a chance to get excited about programming is not just unfortunate, it's deeply harmful, for their future and for ours.
Code Monster and Code Maven from Crunchzilla are designed to make programming easy. Make it fun. Make programming for everyone. In the couple months since launch, they have been used in schools and been getting rave reviews from both girls and boys. One girl "got totally into it" and "when she came up for air", she asked her parents, "Are there jobs you can get working with computers?" And a teacher who used this in a school told me, "A couple 6th grade girls who were not interested in programmers tore through Code Monster then started on Code Academy. It was unexpected and cool!"
If you get a chance to try your children on Code Monster or Code Maven, or you use either in a school, please let me know what you think.
- Google and the right database for the job
I finally got a chance to read "Processing a Trillion Cells per Mouse Click", a paper out of Google presented at the recent VLDB 2012 conference.
It describes the rather cool PowerDrill column-oriented database at Google that is optimized for speed, x10-100 times faster than other column-oriented databases, and several orders of magnitude faster than MapReduce/Hadoop. But, of course, there are tradeoffs to get those speed gains, and the tradeoff PowerDrill makes is that it keeps a lot in memory, so it can only contain a fraction of the data of the other systems.
What is so interesting about this, and what other companies need to learn from this, is the way Google builds so many databases to analyze its massive log data. The goal is to let people find stuff in the logs as fast as possible. That means you need many tools, the right tool for the job.
Hadoop and similar systems allow you to scan massive amounts of log data but, c'mon, all of us know that the vast majority of Hadoop jobs ignore almost all of the data. Every one of these jobs starts by selecting out a couple of the columns, the same columns almost everyone else wants, and dropping everything else. Fire up your job, waste hours of time waiting for almost all the data from a full table scan to be thrown out, and finally you get the result.
Dremel and other column-oriented databases help a lot with this. If almost all log processing jobs only want a couple columns, a column-oriented database is designed to pull out just a few columns quickly, and it's going to be a lot faster.
PowerDrill goes a step further. If almost all log processing jobs only want the most recent logs and only a few of the columns, just create a database with only the most recent logs and a few of the columns. Add in a lot of carefully designed compression, sharding across a medium-sized cluster, and the ability to skip over much of the data when it isn't needed (instead of doing full table scans all the time), and you got yourself the ability to answer most questions people ask of the logs in seconds, not hours.
And that's the point. Build a system that can answer 90% of the questions people ask of the logs in seconds. Build another than can answer 90% of the remaining, harder questions people ask of the logs in minutes. Then have a system that primarily archives all the logs, but also can answer, given enough time and power, much more complicated questions people very rarely ask.
Those Google guys have many databases for asking questions of their logs. Maybe you should too.
Some excerpts from the PowerDrill paper:
The column-store developed as part of PowerDrill is tailored to support a few selected datasets and tuned for speed ... Our column-store relies on having as much data in memory as possible ... PowerDrill can run interactive single queries over more rows than Dremel, however the total amount of data it can serve is much smaller.
Consider a typical use case such as triggering 20 SQL queries with a single click in the UI. In our production system on average these queries process 782 billion cells in 30-40 seconds (under 2 seconds per query) .... Each month it is used by more than 800 users sending out about 4 million SQL queries ... scanning [the equivalent of] 525 trillion cells .... One of our top users ... [in] 6 hours ... [executed about] 12 thousand queries .... Our production system is running on well over 1000 machines, the distributed servers altogether using over 4T of main memory.
[PowerDrill] pushes the "interactivity limit" out significantly ... The majority of queries are fairly discriminative, similar, and uniform ... The store has only a few but often explored tables (as opposed to many tables that are not used very often) ... [For many common queries] our techniques push the limit of interactivity out by one or two orders of magnitude.