naive understanding on research

What is good research: my persepective.

This blog will change from time to time, since I am learning on those time.

The last update: 2023/05/25

Let me first narrow down the topic a little bit, or just take the graph research as an example.

A research paper can be categorized into different types, I will definitely prefer some while not focusing on others. Nonetheles, they are also good research.

An simple birdview can be found as follows:

• Define a new problem either abstract from real-world scenario or focusing on the new ability of new strong model.
• Clear assumption and toy example thelp to explain the intuition. The real-world case is alway complicated, Nonetheless, as a computer scientist, you need to build

To me, I will ask the following question:

Does the paper follows the common evaluation setting?

If I am familar with this research topic, the first thing I will do is to check whether the experimental setting has some tricks. This typically happens on some graph tasks. There could be some specific reasons, however, in most cases, the main reason is that their methods cannot work well. Moreover, it avoid that first believe in the overclaim in most paper, e.g., learn good graph representation for various downstream task, but only has the experiments in node classification.

Does the paper tries to define some new problems?

I think a paper can be a good paper once if it defines a novel problem, It could very hard to define a meaningful task from the real-world scenario. I just take some example.

• In the GPT era, the big company majorly focuses how to train a good LLM model, however, how to find the new potential still need research to develop new research problems to find the ability scopee of the large language model and how to enhance the LLM base on its strong abiallity

If the paper is to propose a new solution to the existing problem, why it is neccerary to propose such method?

In most cases, I have the tolerence that the solution is not good if they define a new problem. However, if it only focuses on solving problem, there are many A+B type paper. Typically in Graph domain, once you see this year ICML, NeurIPS and ICLR, you will know what will next KDD and WWW. As far as I see, I still do not know the reason for utilizing diffusion model for some classification task in graph domain. I do think such paper is good for junior students to learn how to write the paper, make a method works, Nonetheless, there will always be some one who do the same thing with you. If you do not implement this idea, it is very likely that this idea will be done in less than one year. I would like to focus on more important topic that push the domain goes further.

If the paper is a theoritical inspired idea? What is the key intution underlying? Is there a toy example explaining the intuition?