Little has been written on managing your own research (and very little on avoiding other people managing your research); however, your research is much more under your control than you may realize.
We are concerned with great research here. Work that will get wide recognition, perhaps even wine Nobel Prize. As most people realize, the average published paper is read by the author, the referee, and perhaps one other person. Classic papers are read by thousands. We are concerned with research that will matter in the long run and become more than a footnote in history.
If you are to do important work then you must work on the right problem at the right time and in the right way. Without any one of the three, you may do good work but you will almost certainly miss real greatness.
Greatness is a matter of style. For example, after learning the elements of painting, you study under a master. While studying you pay attention to what the master says in discussing your work, but you know that if you are to achieve greatness then you must find your own style. Furthermore, a successful style in one age is not necessarily appropriate for another age. Cubism would not have gone over big during the realism period.
Similarly, there is no simple formula for doing great science or engineering, I can only talk around the topic. The topic is important because, so far as we have any solid evidence, you have but one life to live. Under these circumstances it seems better to live a life in which you do important things (important in your eyes, of course) than to merely live out your life. No sense frittering away your life on things that will not even appear in the footnotes.
I begin with the choice of problem. Most scientists spend almost all of their time working on problems that even they admit are neither great or are likely to lead to great work; hence, almost surely, they will not do important work. Note that importance of the results of a solution does not make the problem important. In all the 30 years I spent at Bell Telephone Laboratories (before it was broken up) no one to my knowledge worked on time travel, teleportation, or anti-gravity. Why? Because they had no attack on the problem. Thus an important aspect of any problem is that you have a good attack, a good starting place, some reasonable idea of how to begin.
To illustrate, consider my experience at BTL. For the first few years I ate lunch with he mathematicians. I soon found that they were more interested in fun and games than in serious work, so I shifted to eating with the physics table. There I stayed for a number of years until the Nobel Prize, promotions, and offers from other companies, removed most of the interesting people. So I shifted to the corresponding chemistry table where I had a friend.
At first I asked what were the important problems in chemistry, then what important problems they were working on, or problems that might lead to important results. One day I asked, "if what they were working on was not important, and was not likely to lead to important things, they why were they working on them?" After that I had to eat with the engineers!
About four months later, my friend stopped me in the hall and remarked that my question had bothered him. He had spent the summer thinking about the important problems in his area, and while had had not changed his research he thought it was well worth the effort. I thanked him and kept walking. A few weeks later I noticed that he was made head of the department. Many years later he became a member of the National Academy of Engineering. The one person who could hear the question went on to do important things and all the others -- so far as I know -- did not do anything worth public attention.
There are many right problems, but very few people search carefully for them. Rather they simply drift along doing what comes to them, following the easiest path to tomorrow. Great scientists all spend a lot of time and effort in examining the important problems in their field. Many have a list of 10 to 20 problems that might be important if they had a decent attack. As a result, when they notice something new that they had not known but seems to be relevant, then they are prepared to turn to the corresponding problem, work on it, and get there first.
Some people work with their doors open in clear view of those who pass by, while others carefully protect themselves from interruptions. Those with the door open get less work done each day, but those with their door closed tend not know what to work on, nor are they apt to hear the clues to the missing piece to one of their "list" problems. I cannot prove that the open door produces the open mind, or the other way around. I only can observe the correlation. I suspect that each reinforces the other, that an open door will more likely lead you and important problems than will a closed door.
Hard work is a trait that most great scientists have. Edison said that genius was 99% perspiration and 1% inspiration. Newton said that if others would work as hard as he did then they would get similar results. Hard work is necessary but it is not sufficient. Most people do not work as hard as they easily could. However, many who do work hard -- work on the wrong problem, at the wrong time, in the wrong way, and have very little to show for it.
You are aware that frequently more than one person starts working on the same problem at about the same time. In biology, both Darwin and Wallace had the idea of evolution at about the same time. In the area of special relativity, many people besides Einstein were working on it, including Poincare. However, Einstein worked on the idea in the right way.
The first person to produce definitive results generally gets all the credit. Those who come in second are soon forgotten. Thus working on the problem at the right time is essential. Einstein tried to find a unified theory, spent most of his later life on it, and died in a hospital still working on it with no significant results. Apparently, he attacked the problem too early, or perhaps it was the wrong problem.
There are a pair of errors that are often made when working on what you think is the right problem at the right time. One is to give up too soon, and the other is to persist and never get any results. The second is quite common. Obviously, if you start on a wrong problem and refuse to give up, you are automatically condemned to waste the rest of your life (see Einstein above). Knowing when you persist is not easy -- if you are wrong then you are stubborn; but if you turn out to be right, then you are strong willed.
I now turn to the major excuse given for not working on important problems. People are always claiming that success is a matter of luck, but as Pasteur pointed out, "Luck favors the prepared mind."
A great deal of direct experience, vicarious experience through questioning others, and reading extensively, convinces me of the truth of his statement. Outstanding successes are too often done by the same people for it be a matter of random chance.
For example, when I first met Feynmann at Los Alamos during the WWII, I believed that he would get a Nobel Prize. His energy, his style, his abilities, all indicated that he was a person who would do many things, and probably at least one would be important. Einstein, around the age of 12 or 14, asked himself what a light wave would look like if he want at the speed of light. He knew that Maxwell's theory did not support a local, stationary maximum, but was what he ought to see if the current theory was correct. So it is not surprising that he later developed the special theory of relativity - he had prepared his mind for it long before.
Many times a discussion with a person who has just done something important will produce a description of how they were led, almost step by step, to the result. It is usually based on things they had done, or intensely thought about, years ago. You succeed because you have prepared yourself with the necessary background long ago, without, of course, knowing then that it would prove to be a necessary step to success.
There traits are not all essential, but tend to be present in most doers of great things in science. First, successful people exhibit more activity, more energy, than most people do. They look more places, they work harder, they think longer than less successful people. Knowledge and ability are much like compound interest -- the more you do the more you can do, and the more the opportunities are open for you. Thus, among other things, it was Feynmann's energy and his constantly trying new things that made one think he would succeed.
This trait must be coupled with emotional commitment. Perhaps the ablest mathematician I have watched up close seldom, if ever, seemed to care deeply about the problem he was working on. He has done great deal of first class work, but not of the highest quality. Deep emotional commitment seems to be necessary for success. The reason is obvious. The emotional commitment keeps you thinking about the problem morning, noon and night, and that tends to beat out mere ability.
While I was at Los Alamos after the war, I got to thinking about the famous Buffon needle problem where you can calculate the probability of a needle tossed at random of crossing one of a series of equally spaced parallel lines. I asked myself if it was essential that the needle be a straight line segment (if I counted multiple crossing)? No. Need the parallel lines be straight? No. Need they be equally spaced or is it only the average density of the lines on the plane? Is it surprising that some years later at Bell Labs when I was asked by some metallurgists how to measure the amount of grain boundary on some micro photographs I simply said, "Count the crossings of a random line of fixed length on the picture?" I was led to it by the previous, careful thought about an interesting, and I thought important, result in probability. The result is not great, but illustrates the mechanisms of preparation and emotional involvement.
The above story also illustrates what I call the "extra mile." I did more than the minimum, I looked deeper into the nature of the problem. This constant effort to understand more than the surface feature of a situation obviously prepares you to see new and slightly different applications of your knowledge. You cannot do many problems such as the above needle problem before you stumble on an important application.
Courage is another attribute of those who do great things. Shannon is a good example. For some time he would come to work at about 10:00am, play chess until about 2:00pm and go home.
The important point is how he played chess. When attacked he seldom, if ever, defended his position, rather he attacked back. Such a method of playing soon produces a very interrelated board. He would then pause a bit, think and advance his queen saying, "I ain't afraid of nothing'." It took me a while to realize that of course that is why he was able to prove the existence of good coding methods. Who but Shannon would think to average over all random codes and expect to find that the average was close to ideal? I learned from him to say the same to myself when stuck, and on some occasions his approach enabled me to get significant results.
Without courage you are unlikely to attack important problems with any persistence, and hence not likely to do important things. Courage brings self-confidence, an essential feature of doing difficult things. However, it can border on over-confidence at time which is more of a hindrance than a help.
There is another trait that took me many years to notice, and that is the ability to tolerate ambiguity. Most people want to believe what they learn is the truth: there are a few people who doubt everything. If you believe too much then you are not likely to find the essentially new view that transforms a field, and if you doubt too much you will not be able to do much at all. It is a fine balance between believing what you learn and at the same time doubting things. Great steps forward usually involve a change of viewpoint to outside the standard ones in the field.
While you are leaning things you need to think about them and examine them from many sides. By connecting them in many ways with what you already know.... you can later retrieve them in unusual situations. It took me a long time to realize that each time I learned something I should put "hooks" on it. This is another face of the extra effort, the studying more deeply, the going the extra mile, that seems to be characteristic of great scientists.
The evidence is overwhelming that steps that transform a field often come from outsiders. In archaeology, carbon dating came from physics. The first airplane was built by the Wright brothers who were bicycle experts.
Thus, as an expert in your field, you face a difficult problem. There is, apparently, an ocean of kooks with their crazy ideas; however, if there is a great step forward it probably will be made by one of them! If you listen too much to them then you will not get any of your own work done, but if you ignore them then you may miss your great chance. I have no simple answer except do not dismiss the outsider too abruptly as is generally done by in the insiders.
"Brains"" are nice to have, but often the top graduate students do not contribute as much as some lower rated ones. Brains come in all kinds of flavors. Experimental physicists do not think the same way as theoreticians do. Some experimentalists seem to think with their hands, i.e., playing with equipment lets them think more clearly. It took me a few years to realize that people who did not know a lot of mathematics still could contribute. Just because they could not solve a quadratic equation immediately in their head did not mean I should ignore them. When someone's flavor of brains does not match yours may be more reason for paying attention to them.
You need a vision of who you are and where your field is going. A suitable parable is that of the drunken sailor. He staggers one way and then the other with independent, random steps. In n steps he will be, on the average, about 3n steps away from where he started. but if there is a pretty girl in one direction he will get a distance proportional to n. The difference, over a life time of choices, between 3n and n is very large and represents the difference between having no vision and having a vision. The particular vision you have is less important than just having one - there are many paths to success. Therefore, it is wise to have a vision of what you may become, of where you want to go, as well as how to get there. No vision, not much chance of doing great work; with a vision you have a good chance.
Another topic I must discuss is that of age. Historically, the greatest contributions of mathematicians, theoretical physicists, and astrophysicists are done when they are very young. On the other hand, apparently in music composition, politics, and literature, the later works are most valued by society. Other areas seem to fall in between these extremes, and you need to realize that in some areas you had better get going promptly.
People often complain about the working conditions they have to put up with, but it is easily observed that some of the greatest work was done under unfavorable conditions. What most people believe is the best working conditions for them is seldom, if ever, true. In my opinion the Institute for Advanced Study in Princeton has ruined more good people than it has helped. You have only to judge their work before they were appointed and afterwards to come to this conclusion. There are exceptions, to be sure, but on the average the supposed ideal working conditions seem to sterilize people.
Another obvious trait of great people is that they do their work in such a fashion that others can build on top of it. Newton said, "If I had seen farther than others it is because I stood on the shoulders of giants." Too many people seem to not want others to build on top of their work but rather they want to hoard it to themselves. Don't do things in such a fashion that next time it must be repeated by you, or by others, but rather in a fashion that represents a significant step forward.
I must now take up the unpleasant topic of selling your ideas. Too many scientists think that this is beneath them, that the world is waiting for their great results. In truth, the other researchers are busy with their own work. You must present your results so that they will stop their own work and listen to you. Presentation comes in three forms: published papers, prepared talks, and impromptu situations. You must master all three forms.
Lots of good work has been lost because of poor presentation only to be rediscovered later by others. There is a real danger that you will not get credit for what you have done. I know of all too many times when the discoverer could not be bothered to present things clearly, and hence his or her work was of no importance to society.
Finally, I must at least address the question of whether greatness is worth the large effort it requires. Those who have done really great things generally report, privately, that it is better than wine, the opposite sex, and song put together. The realization that you have done it is overwhelming.
Of course I have consulted only those who did do great things, and have no dared to ask those who did not. Perhaps they would reply differently. But, as is often said, it is in the struggle and not the success that the real gain appears. In striving to do great things, you change yourself into a better person, so they claim. The actual success is of less importance, so they say. And I tend to believe this theory.
No one ever told me the kinds of things I have just related to you; I had to find them out for myself. Since I have now told you how to succeed, you have no excuse for not trying and doing great work in your chosen field.
Dr. Richard Hamming is best known for the Hamming code, Hamming distance and the Hamming spectral window along with numerical methods.