Entering the Academic Conversation, Seriously

When you write a research essay, you’re joining an ongoing conversation. If, for example, someone writes about climate change, the author is stepping into a conversation that has been happening for the past fifty years and longer.

Part of joining that conversation means being able to find and evaluate which voices matter for the paper’s topic and thesis, and then building upon and responding to those people.

The twentieth century rhetorician Kenneth Burke posits there is an “unending conversation” occurring throughout history, and his illustration is helpful in understanding how your writing will join a conversation of ideas:

Imagine that you enter a parlor. You come late. When you arrive, others have long preceded you, and they are engaged in a heated discussion, a discussion too heated for them to pause and tell you exactly what it is about. In fact, the discussion has already begun long before any of them got there, so that no one present is qualified to retrace for you all the steps that had gone before. You listen for a while, until you decide that you have caught the tenor of the argument; then you put in your oar. Someone answers; you answer him; another comes to your defense; another aligns himself against you, to either the embarrassment or gratification of your opponent, depending upon the quality of your ally’s assistance. However, the discussion is interminable. The hour grows late, you must depart. And you do depart, with the discussion still vigorously in progress.

In this analogy, the other party-goers are the authors of primary and secondary texts an author engages in a paper, and the person writing the research paper is the one joining the conversation. However, the student in this case has many more sources to choose from than just a few in a single conversation. There are many voices in hundreds of conversations, and it is up to you to choose which ones are most important and worth including in your argument.

Evaluating the Important Voices

In 2010, a textbook being used in fourth grade classrooms in Virginia became big news for all the wrong reasons. According to an article in The Washington Post, the book Our Virginia, by Joy Masoff, had caught the attention of a parent who was helping her child do her homework. Carol Sheriff was a historian for the College of William and Mary and as she worked with her daughter using Our Virginia, she began to notice some glaring historical errors,

not the least of which was a passage which described how thousands of African Americans fought for the South during the Civil War.

Further investigation into the book revealed that although the author had written textbooks on a variety of subjects, she was not a trained historian. The research she had done to write Our Virginia, and in particular the information she included about Black Confederate soldiers, was done through the internet and included sources created by groups like the Sons of Confederate Veterans, an organization which promotes views of history that de-emphasize the role of slavery in the Civil War.

How did a book with errors like these come to be used as part of the curriculum and who was at fault? Was it Masoff for using untrustworthy sources for her research? Was it the editors who allowed the book to be published with these errors intact? Was it the school board for approving the book without more closely reviewing its accuracy?

There are a number of issues at play in the case of Our Virginia, but there’s no question that evaluating sources is an important part of the research process and doesn’t just apply to internet sources. Using inaccurate, irrelevant, or poorly researched sources can affect the quality of your own work. Being able to understand and apply the concepts that follow is crucial to becoming a more savvy user and creator of information.

People who are adept at evaluating information are able to review the research process and compare and judge information and data. Individuals who evaluate information well:

Understand:

• The information and data landscape of their learning/research context;
• Issues of quality, accuracy, relevance, bias, reputation and credibility relating to information and data sources;
• How information is evaluated and published, to help inform their personal evaluation process;
• The importance of consistency in data collection;
• The importance of citation in their learning/research context.

Are able to:

• Distinguish between different information resources and the information they provide;
• Choose suitable material on their search topic using appropriate criteria;
• Assess the quality, accuracy, relevance, bias, reputation and credibility of sources;
• Assess the credibility of the data gathered;
• Read critically, identifying key points and arguments;
• Relate the information found to the original search strategy;
• Critically appraise and evaluate their own findings and those of others;
• Know when to stop.

Distinguishing Between Information Resources

Information is published in a variety of formats, each with its own special considerations when it comes to evaluation. When conducting your research, consider the following formats and the different ways in which each format is developed and presented:

Social Media

Social media is a quickly rising star in the landscape of information gathering. Facebook updates, tweets, wikis, and blogs have made information creators of us all and have a strong influence not just on how we communicate with each other but also on how we learn about current events or discover items of interest. Anyone can create or contribute to social media and nothing that’s said is checked for accuracy before it’s posted for the world to see. So do people really use social media for research? Currently, the main use for social media, like tweets and Facebook posts, is as primary sources that are treated as the objects under study rather than sources of information on a topic.

Websites

In addition to social media, a common place to find information is websites. Commercial organizations, educational institutions, government entities, research think tanks, and more publish blogs, reports, and statistics that could be relevant to your research. However, evaluating the credibility of web sources can be challenging. Students are often taught in high school, as a general rule of thumb, to avoid .com websites as untrustworthy and to use .org and .edu domains instead. In research, it is important to critically evaluate all web sources for trustworthiness, accuracy, and currency, regardless of domain suffix. Websites are considered popular sources, even if published by an educational institution or a government entity. Some websites, including some government websites, may be considered primary sources, which could have value in research. (You’ll find much more about this later in this chapter.)

News Articles

These days, social media will sometimes be among the first to cover a big news story, with news media writing an article or report after more information has been gathered. News articles are written by journalists who either report on an event they have witnessed firsthand, or after making contact with those more directly involved. The focus is on information that is of immediate interest to the public and these articles are written in a way that a general audience will be able to understand. These articles go through a fact-checking process, but when a story is big and the goal is to inform readers of urgent or timely information, inaccuracies may occur. In research, news articles are often best treated as primary sources, especially if they were published immediately after a current event.

Magazine Articles

While social media tend to concentrate on what happened, how it happened, who it happened to, and where it happened, magazine articles are often more about understanding why something happened, usually with the benefit of at least a little hindsight. Writers of magazine articles also fall into the journalist category and rely heavily on investigation and interviews for research. Fact-checking in magazine articles can be more rigorous because magazines publish less frequently than news outlets and have more time to get the facts right. (Though in the digital world, that’s not necessarily so. Magazines, like newspapers and other new outlets, publish virtually continuously.) Depending on the focus of the magazine, articles may cover current events or just items of general interest to the intended audience. A magazine article is considered a popular source rather than a scholarly one which gives it less weight in a research context but doesn’t take away the value completely.

Scholarly Articles

Scholarly articles are written by, and for, experts in a field and generally describe formal research studies or experiments conducted to provide new insight on a topic rather than reporting current events or items of general interest. You may have heard the term “peer review” in relation to scholarly articles. This means that before an article is published, it undergoes a review process to confirm that the information is accurate and the research it discusses is valid. This process adds a level of credibility to the article that you would not find in a magazine or news article. Scholarly articles tend to be long and feature specialized language – jargon — that is not easily understood by someone who does not already have some level of expertise on the topic. Though they may not be as easy to use, they carry a lot of weight in a research context, especially if you are working in a field related to science or technology. These sources will give you information to build on in your own original research. (You’ll find much more about scholarly articles later in this chapter.)

Books

Books have been a staple of the research process since Gutenberg invented the printing press because a topic can be covered in more depth in a book than in most other types of sources. Also, the conventional wisdom for books is that anyone can write one, but only the best ones get published. This is becoming less true as books are published in a wider variety of formats and via a wider variety of venues than in previous eras, which is something to be aware of when using a book for research purposes. For now, the editing process for formally published books is still in place and research in the humanities, which includes topics such as literature and history, continues to be published primarily in this format.

Choosing Materials

Here are some criteria to keep in mind when deciding if a source will be useful to your research.

Quality

Scholarly journals and books are traditionally considered to be higher quality information sources because they have gone through a more thorough editing process that ensures the quality of their content. Generally, you also pay more to access these sources or may have to rely on a library or university to pay for access for you. Information on the Internet can also be of a high quality but there is less of a quality assurance process in place for much of that information. In the current climate, the highest quality information, even on the Internet, often requires a subscription or other form of payment for access.

Clues to a source’s level of quality are closely related to thinking about how the source was produced, including what format it was published in and whether it is likely to have gone through a formal editing process prior to publication.

Accuracy

A source is accurate if the information it contains is correct. Sometimes it’s easy to tell when a piece of information is simply wrong, especially if you have some prior knowledge of the subject. But if you’re less familiar with the subject, inaccuracies can be harder to detect, especially when they come in subtler forms such as exaggerations or inconsistencies.

To determine whether a source is accurate, you need to look more deeply at the content of the source, including where the information in the source comes from and what evidence the author uses to support their views and conclusions. It also helps to compare your source against another source. A reader of Our Virginia may not have reason to believe the information the author cites from the Sons of Confederate Veterans website is inaccurate, but if they compared the book against another source, the inconsistencies might become more apparent.

Relevance

Relevance has to do with deciding whether the source actually relates to your topic and, if it does, how closely it relates. Some sources may be an exact match; for others, you may need to consider a particular angle or context before you can tell whether the source applies to your topic. When searching for relevant sources, you should keep an open mind—but not too open. Don’t pick something that’s not really related just because it’s on the first page or two of results or because it sounds good.

You can assess the relevance of a source by comparing it against your research topic or research question. Keep in mind that the source may not need to match on all points, but it should match on enough points to be usable for your research beyond simply satisfying a requirement for an assignment.

Bias

An example of bias is when someone expresses a view that is one-sided without much consideration for information that might negate what they believe. Bias is most prevalent in sources that cover controversial issues where the author may attempt to persuade their readers to one side of the issue without giving fair consideration to the other side of things. If the research topic you are using has ever been the cause of heated debate, you will need to be especially watchful for any bias in the sources you find.

Bias can be difficult to detect, particularly when we are looking at persuasive sources with which we want to agree. If you want to believe something is true, chances are you’ll side with your own internal bias without consideration for whether a source exhibits bias. When deciding whether there is bias in a source, look for dramatic language and images, poorly supported evidence against an opposing viewpoint, or a strong leaning in one direction.

Reputation

Is the author of the source you have found a professor at a university or a self-published blogger? If the author is a professor, are they respected in their field or is their work heavily challenged? What about the publication itself? Is it held in high regard or relatively unknown? Digging a little deeper to find out what you can about the reputation of both the author and the publication can go a long way toward deciding whether a source is valuable.

You can investigate the reputation of an author by looking at any biographical information that is available as part of the source. Looking to see what else the author has published and whether this information has positive reviews is also important in establishing whether the author has a good reputation. The reputation of a publication can also be investigated through reviews, word-of-mouth by professionals in the field, or online databases that keep track of statistics related to a journal’s credibility.

Credibility

Credibility has to do with the believability or trustworthiness of a source based on evidence such as information about the author, the reputation of the publication, and how well the source is formatted. How likely would you be to use a source that was written by someone with no expertise on a topic or a source that appeared in a publication that was known for featuring low quality information? What if the source was riddled with spelling and formatting errors? Looking at sources like these should inspire more caution.

Objectively, credibility can be determined by taking into account all of the other criteria discussed for evaluating a source. Knowing that some types of sources, such as scholarly journals, are generally considered more credible than others, such as self-published websites, may also help. Subjectively, deciding whether a source is credible may come down to a gut feeling. If something about a source doesn’t sit well with you, you may decide to pass it over.

Identifying Key Points and Arguments

Evaluating information about the source from its title, author, and summary information is only the first step. The evaluation process continues when you begin to read the source in more detail and make decisions about how (or whether) you will ultimately use it for your own research.

When you begin to look more deeply at your source, pay close attention to the following features of a document.

Introduction

The purpose of the introduction to any piece that has one is to give information about what the reader can expect from the source as a whole. There are different types of introductions, including forewords and prefaces that may be written by the author of the book or by someone else with knowledge of the subject. Introductory sections can include background information on why the topic was chosen,

background on the author’s interest in the topic, context pertaining to why the topic is important, or the lens through which the topic will be explored. Knowing this information before diving into the body of the work will help you understand the author’s approach to the topic and how it might relate to the approach you are taking in your own research.

Table of Contents

Most of the time, if your source is a book or an entire website, it will be divided into sections that each cover a particular aspect of the overall topic. It may be necessary to read through all of these sections in order to get a “big picture” understanding of the information being discussed, or it may be better to concentrate only on the areas that relate most closely to your own research. Looking over the table of contents or menu will help you decide whether you need the whole source or only pieces of it.

List of References

If the source you’re using is research-based, it should have a list of references that usually appear at the end of the document. Reviewing these references will give you a better idea of the kind of work the author put into their own research. Did they put as much work into evaluating their sources as you are? Can you tell from the citations if the sources used were credible? When were they published? Do they represent a fair balance of perspectives or do they all support a limited point of view? What information does the author use from these sources and in what way does he or she use that information? Use your own research skills to spy on the research habits of others to help you evaluate the source.

Evaluating Your Findings

In the case of Our Virginia, the author used a biased source as part of her research and the inaccurate information she got from that source affected the quality of her own work. Likewise, if anyone had used her book as part of their research, it would have set off a chain reaction, since whatever information they cited from Our Virginia would naturally have to be called into question, possibly diminishing the value of their own conclusions.

Evaluating the sources you use for quality, accuracy, relevance, bias, and credibility is a good first step in making sure this doesn’t happen, but have you ever thought about evaluating the sources used by your own sources? This takes extra time, but looking at the reference list, bibliography, or notes section of any source you use to gauge the quality of the research done by the author of that source can be an important extra step.

Knowing When to Stop

For some researchers, the process of searching for and evaluating sources is a highly enjoyable, rewarding part of doing research. For others, it’s a necessary evil on the way to constructing their own ideas and sharing their own conclusions. Whichever end of the spectrum you most closely identify with, here are a few ideas about the ever-important skill of knowing when to stop.

1. You’ve satisfied the requirements for the assignment or your curiosity on the topic.

If you’re doing research as part of a course assignment, chances are you’ve been given a required number of sources. Novice researchers may find this number useful to understand how much research is considered appropriate for a particular topic. However, a common mistake is to focus more on the number of sources than on the quality of those sources. Meeting that magic number is great, but not if the sources used are low quality or otherwise inappropriate for the level of research being done.

2. You have a deadline looming.

Nothing better inspires forward motion in a research project than having to meet a deadline, whether it’s set by a professor, an advisor, a publisher, or yourself. Time management skills are especially useful, but since research is a cyclical process that sometimes circles back on itself when you discover new knowledge or change direction, planning things out in minute detail may not work. Leaving yourself enough time to follow the twists and turns of the research and writing process goes a long way toward getting your work in when it’s expected.

3. You need to change your topic.

You’ve been searching for information on your topic for a while now. Every search seems to come up empty or full of irrelevant information. You’ve brought your case to a research expert, like a librarian, who has given advice on how to adjust your search or how to find potential sources you may have previously dismissed. Still nothing. It could be that your topic is too specific or that it covers something that’s too new, like a current event that hasn’t made it far enough in the information cycle yet. Whatever the reason, if you’ve exhausted every available avenue and there truly is no information on your topic of interest, this may be a sign that you need to stop what you’re doing and change your topic. Sometimes, when students reach this point, instead of changing their topics, they lower their standards, turning to the internet to find something – anything – that supports their theses. Do not do this. No matter how wrong, no matter how off-the-wall a thesis is, somewhere on the internet someone on the internet someone has almost certainly written something that supports it. Do not relax your standards of credibility, quality and accuracy to avoid changing topics. If you need to change your topic, of course, it’s always best to do that early in the process. Any time you spend on a dead-end topic is time you won’t be able to spend on the paper you’ll eventually turn in to your instructor.

4. You’re getting overwhelmed.

The opposite of not finding enough information on your topic is finding too much. You want to collect it all, read through it all, and evaluate it all to make sure you have exactly what you need. But now you’re running out of room on your flash drive, your Dropbox account is getting full, and you don’t know how you’re going to sort through it all and look for more. The solution: stop looking. Go through what you have. If you find what you need in what you already have, great! If not, you can always keep looking. You don’t need to find everything in the first pass. There is plenty of opportunity to do more if more is needed.

From Theory to Practice

Looking back, the Our Virginia case is more complicated than it may have first appeared. It wasn’t just that the author based her writing on research done through the Internet. It was complicated by the nature of the sources she used as well as the effect using those sources ultimately had on the quality of her own work. These mistakes happened despite a formal editing process that should have ensured better accuracy and an approval process by the school board that should have evaluated the material more closely. With both of these processes having failed, it was up to one of the book’s readers, the parent of a student who compared the information against her own specialized knowledge, to figure it all out.

Now that you know more about the theory behind evaluating sources, it’s time to apply the theory. The following section will help you put source evaluation into perspective using something you probably remember from elementary school – the five Ws and H.

The Five Ws

You evaluate sources regularly in your day-to-day life. You evaluate which movie to see this weekend, which new restaurant to try out, and which section of a required college class to take next semester. Evaluating sources in an academic context requires many of the same skills you already employ.

When you begin evaluating sources, what should you consider? There is no single test or checklist that you can use to determine if a source is trustworthy or appropriate. Evaluating sources requires critical examination of each source, as well as the context in which you intend to use that source.

When evaluating sources, one set of criteria that you should consider are the five Ws and H. These questions can help you begin to suss out not only the quality of your source, but whether it is appropriate for your intended use case.

Who

Understanding more about your information’s source helps you determine when, how, and where to use that information. Is your author an expert on the subject? Do they have some personal stake in the argument they are making? What is the author or information producer’s background? When determining who is behind your source, consider the following:

• Who is the author? What are their credentials?
• What is the author’s level of education, experience, and/or occupation?
• What qualifies the author to write about this topic?
• What affiliations does the author have? Could these affiliations affect their position?
• What organization or body published the information? Is it authoritative? Does it have an explicit position or bias? Is it scholarly?
• Was this information funded by a group or organization? If so, who funded it? Is there a conflict of interest or perception of bias?

What

Knowing the intended purpose of a piece of information is a key to evaluation. Understanding whether the information is fact, opinion or propaganda will help you decide whether and how to use it. When determining what is the purpose of your source, consider the following:

• What information does the author present?
• Is the information clearly supported by evidence?
• Does the source have an obvious bias or prejudice?
• What information isn’t presented? Does the author omit important facts or data that might disprove their argument?
• Is the article presented from multiple points of view?
• Is the author’s language informal, joking, emotional, or impassioned?
• Is the source intended to inform, sell, persuade, or entertain?

Where

Determining where information comes from, if the evidence supports the information, and if the information has been reviewed or refereed can help you decide how and whether to use a source. When determining from where a source derives its information, consider the following:

• Where does the information in the article come from?
• Where can I find more information about the topic?
• Is the source well-documented? Does it include footnotes, citations or a bibliography?
• Is information in the source presented as fact, opinion or propaganda? Are biases clear?
• Can you verify information from referenced information in the source?

When

One of the most important and interesting steps to take as you begin researching a subject is selecting the resources that will help you build your thesis and support your assertions. Many topics require you to pay special attention to how current your resource is—because they are time sensitive, because they have evolved so much over the years, or because new research comes out on the topic so frequently.

When evaluating the currency of an article, consider the following:

• When was the item written and how frequently does the publication it is in come out?
• Is there evidence of newly added or updated information in the item?
• If the information is dated, is it still suitable for your topic?
• How frequently does information change about your topic?
• Has the information been updated recently?

Why

As you consider each source, consider the sheer quantity of information available to you. A simple Google search may retrieve billions of search results in response to a keyword search on your topic. Why is this source the best one to use for your particular topic and project? Remember that you’re not looking for the first source you can find on a topic – you’re looking for the one that is going to provide the strongest evidence for your information need. When considering why you should use a particular source, consider the following:

• Why is this information better than other available sources?
• Why is this information considered high quality, scholarly, or trustworthy?
• Why does this article support or undermine your research hypothesis?
• Is the information written clearly and free of typographical and grammatical mistakes? Does the source look to be edited before publication? A clean, well-presented paper does not always indicate accuracy, but usually at least means more eyes have been on the information.

How

Understanding what resources are most applicable to your subject and why they are applicable can help you focus and refine your thesis. Many topics are broad and searching for information on them produces a wide range of resources. Narrowing your topic and focusing on resources specific to your needs can help reduce the piles of information and help you focus in on what is truly important to read and reference. When determining how a source may be relevant to your topic, consider the following:

• How does this relate to what you already know?
• How do you intend to use this information?
• Does the item contain information relevant to your argument or thesis?
• Does the information presented support or refute your ideas?
• If the information refutes your ideas, how will this change your argument?

  Using the Right Sources for Your Project

Selecting the right type of source for your information is an important aspect of researched writing. The right source can provide evidence and add credibility to your position, but the wrong source   can undermine your position. The context of your project is key to choosing the right source; even a completely specious source can be the right source if it’s evidence in support of a paper about specious sources. For most of your projects, though, you’ll likely be looking for high-quality sources, especially scholarly sources.

Types of Evidence in Academic Arguments

All academic writers use evidence to support their claims. However, as writing tasks vary across disciplinary fields, they require different types of evidence. Often, it takes a combination of different types of evidence to adequately support and develop a point.

To clarify, evidence is what a writer uses to support or defend the writer’s argument, and only valid and credible evidence is enough to make an argument strong.

Evidence is not simply “facts.” Evidence is not simply “quotes.”

 As you develop your research-supported essay, consider not only what types of evidence might support your ideas but also what types of evidence will be considered valid or credible according to the academic discipline or academic audience for which you are writing. The following are some examples of credible evidence by academic discipline.

Evidence in the Humanities: Literature, Art, Film, Music, Philosophy

• Scholarly essays that analyze original works
• Details from an image, a film, or other work of art
• Passages from a musical composition
• Passages of text, including poetry

Evidence in the Humanities: History

• Primary Sources (photos, letters, maps, official documents, etc.)
• Other books or articles that interpret primary sources or other evidence

Evidence in the Social Sciences: Psychology, Sociology, Political Science, Anthropology

• Books or articles that interpret data and results from other people’s original experiments or studies.
• Results from one’s own field research (including interviews, surveys, observations, etc.)
• Data from one’s own experiments
• Statistics derived from large studies

Evidence in the Sciences: Biology, Chemistry, Physics

• Data from the author of the paper’s own experiments
• Books or articles that interpret data and results from other people’s original experiments or studies.

What remains consistent no matter the discipline in which you are writing, however, is that evidence never speaks for itself. Quality evidence must be integrated it into your own argument or claim to demonstrate the evidence supports your thesis. In addition, be alert to evidence that seems to contradict your claims or offers a counterargument to it. Rebutting that counterargument can be powerful evidence for your claim. You can also make evidence that isn’t there an integral part of your argument, too. If you can’t find the evidence you think you need, ask yourself why it seems to be lacking, or if its absence adds a new dimension to your thinking about the topic. Remember, evidence is not the piling up of facts or quotes. Evidence is only one component of a strong, well supported, well argued, and well written composition.

Scholarly Sources

While reading academic articles can be one of the more intimidating aspects of college- level research projects, several aspects to the purpose, format, and style of academic journal articles are rather straightforward and patterned. Knowing the template scholarly articles follow can enhance your comprehension and make these intimidating reading materials much less daunting. Moreover, understanding the purpose of scholarly publication can help you understand what matters most in these articles.

The term scholarly can be quite confusing, as some professors will use it broadly to mean any source that is reputable and academic in nature, while other professors will use it more narrowly to refer specifically to sources that have gone through a peer review process. Generally speaking, sources fall on a spectrum between popular and scholarly, with peer-reviewed research articles and monographs fitting most clearly under the scholarly category. Always

check with your professor to be sure how they are defining scholarly sources for the purposes of your class.

Basic Format

Information in academic journal articles is presented in a formal, highly prescribed format, meaning that scholarly articles tend to follow a similar layout, pattern, and style. The pages often look stark, with little decoration or imagery. The article title is often fairly prominent on the first page, as are the author(s)’ name(s). Sometimes there is a bit of information about each author, such as the name of his or her current academic institution or academic credentials. At either the top or bottom of the first few pages, you can find the name of the scholarly journal in which the article is published.

Abstract

On the first page of the article, you will often find an abstract, which is a summary of the author’s research question, methodologies and results. While this abstract is useful to you as a reader because it gives you some background about the article before you begin reading, you should not cite this abstract in your paper. Please read these abstracts as you are initially seeking sources so you can determine whether reading the article will be useful to you, but do not quote or paraphrase from the abstract.

Works Cited

At the end of academic articles, you will find a list of works cited (sometimes called a list of references). This is generally quite long, and it details all of the work the author cited in the article.

Helpful hint: reading the works cited in an article you find particularly illuminating or useful can be a great way to locate other sources that may be useful for your own research project! If you see a title that looks interesting, see if you can access it through your university’s library.

Literature Review

Scholarly sources often contain literature reviews in the beginning section of the article. They are generally several paragraphs or pages long. Some articles are only literature reviews. These literature reviews generally do not constitute an author’s own work. Instead, they are summaries and syntheses of other scholars’ work that has previously been published on the topic the author is addressing in this paper. Including this review of previous research helps communicate the author’s understanding of the context out of which the author’s research comes.

Like the abstract, the literature review is another part of a scholarly article from which you should generally not quote. Doing so is sort of like citing a SparkNotes version of an essay you have not read. The literature review is where your author describes previous research. He or she is outlining what others have said in their own articles, not offering a new insight (and what we are interested in in scholarly articles is the new information a researcher brings to the topic). If you find that there is interesting information from the sources that your author discusses in the literature review, then you should find the article(s) the author is summarizing and read them for yourself. That, in fact, is a great strategy for finding more sources.

The “Research Gap”

Somewhere near the end of the literature review, authors may indicate what has not been said or not been examined by previous scholars. Some call this a “research gap” – a space out of which a scholar’s own research develops. The “research gap” opens the opportunity for the author to ask a question or assert a claim. Academic authors who want to publish in scholarly research journals need to define a research gap and then attempt to fill that gap because scholarly journals want to publish new, innovative and interesting work that will push knowledge and scholarship in that field forward. Scholars must communicate what new ideas they have worked on: what their new hypothesis, or experiment, or interpretation or analysis is. (Some people have suggested this creates problems, since much more energy is devoted to “new, innovative and interesting work” rather than replicating – in a sense, fact checking – previous work. You’ll find an interesting exploration of this at this link.)

The Scholar Adds a New Perspective

Then, and sometimes for the bulk of an academic article, authors discuss their original

work and analysis. This is the part of the article where authors add to the conversation, where they try to fill in the research gap they identified. This is also the part of the article that is the primary research. Authors may include a discussion of their research methodology and results, or an elaboration and defense of their reasoning, interpretation or analysis. Scholarly articles in the sciences or social sciences may include headings such as “Methods,” “Results,” and “Discussion” or synonyms of those words in this part of the article. In arts or humanities journal articles, these headings may not appear because scholars in the arts and humanities do not necessarily perform lab-based research in the same way as scientists or social scientists do. Authors may reference others’ research even in this section of original work and analysis, but only to support or enhance the discussion of the scholar’s own discussion (much as you’ll do in your papers). This is the part of the scholarly article you should cite from, as it indicates the work the authors you’re relying on  have done.

Conclusion

To conclude a scholarly journal article, authors may reference their original research question or hypothesis once more. They may summarize some of the points made in the article. We often see scholars concluding by indicating how, why, or to whom their research matters. Often, authors will conclude by looking forward, offering ideas for other scholars to engage in future research. Sometimes, they may reflect on why an experiment failed (if it did) and how to approach that experiment differently next time. What we do not tend to see is scholars merely summarizing everything they discussed in the essay, point by point. Instead, they want to leave readers with a sense of why the work they have discussed in their article matters.

As you read scholarly sources, remember

• to look for the author’s research question or hypothesis
• to seek out the “research gap”: why did the author have this research question or hypothesis?
• to identify the Literature Review
• to identify the point at which the author stops discussing previous research and begins to discuss his or her own
• Most importantly: remember to always try to understand what new information this article brings to the scholarly “conversation” about this topic?

Types of Sources: Primary, Secondary, Tertiary

The determination of a text as “popular” or “scholarly” is one way to classify it and to understand what type of information you are engaging with. Another way to classify sources is by considering whether they are primary, secondary or tertiary. Popular sources can be primary, secondary, or tertiary. Scholarly sources, also, can be primary, secondary, or tertiary.

What is a Primary Source?

• Primary sources are texts that arise directly from a particular event or time period. They may be letters, speeches, works of art, works of literature, diaries, direct personal observations, newspaper articles that offer direct observations of current events, survey responses, tweets, other social media posts, original scholarly research (meaning research that the author or authors conduct themselves) or any other content that comes out of direct involvement with an event or a research study.
• Primary research is information that has not yet been critiqued, interpreted or analyzed by a second (or third, etc.) party.
• Primary sources can be popular (if published in newspapers, magazines or websites for the general public) or academic (if written by scholars and published in scholarly journals).

Examples of primary sources include:

• journals, diaries
• blog posts
• a speech
• data from surveys or polls
• scholarly journal articles in which the author(s) discuss the methods and results from their own original research/experiments
• photos, videos, sound recordings
• interviews or transcripts
• poems, paintings, sculptures, songs or other works of art
• government documents (such as reports of legislative sessions, laws or court decisions, financial or economic reports, and more)

What is a Secondary Source?

• Secondary sources summarize, interpret, critique, analyze, or offer commentary on primary sources.
• In a secondary source, an author’s subject is not necessarily something that he or she directly experienced. The author of a secondary source may be summarizing, interpreting or analyzing data or information from someone else’s research or offering an interpretation or opinion on current events. Thus, the secondary source is one step away from that original, primary topic/subject/research study.
• Secondary sources can be popular (if published in newspapers, magazines or websites for the general public) or academic (if written by scholars and published in scholarly journals).

Examples of secondary sources include:

• book, movie or art reviews
• summaries of the findings from other people’s research
• interpretations or analyses of primary source materials or other people’s research
• histories or biographies
• political commentary

What is a Tertiary Source?

Tertiary sources are syntheses of primary and secondary sources. The person/people who compose a tertiary text are summarizing, compiling, and/or paraphrasing others’ work. These sources sometimes do not list an author. Tertiary sources can be popular or academic.

Examples of tertiary sources include:

• encyclopedias
• fact books
• dictionaries
• guides
• handbooks
• Wikipedia

Finding Accurate Information on the Internet

For most academic papers, you’ll want to depend as much as possible on academic journals – peer-reviewed publications – for evidence and information. Peer-reviewed publications are pretty much what they sound like. Before those publications distribute an article, that article is reviewed by authorities in whatever discipline the article addresses. Those are the gold standard of sources. The problem is not every topic you’ll write about or wonder about has been covered by peer-reviewed journals. Besides, once you’re out of college, it may not be so easy to access all those expensive databases available to you now through the Brown Library. You’ll most likely do what you do most of the time already – Google. Through Google – and Siri and Alexa and all the other devices, voices and search engines available – any person with access to a decent WiFi connection has more information more easily available than anyone in any of the 6,666 or so preceding human generations.¹ Some of that information is garbage. Some of it is advertising disguised as objective information. Some of it is solid, useful information. The challenge is to tell the difference.

“Any sufficiently advanced technology is indistinguishable from magic.”

-Arthur C. Clarke

Profiles of the Future: An Inquiry into the Limits of the Possible

Search engines can seem like magic. Speak or type a question, or punch some key words into a search line, and less than a second later, answers appear. Often, millions of answers appear. Those at the top are usually there because someone paid for them to be there. The next in line are often there because they’re popular; lots of people have clicked on them before. Many are there because an algorithm decided the person doing the searching might be interested in them; the searcher has spent time on similar topics before. That may be a good thing if the search is about music or shoes or a funny memes about college professors. But returning to old topics and sources can also lead to dark and misleading places. A Southern Poverty Law Center video, for example, illustrates how Dylan Roof began by searching “black on white crime” and ended up murdering nine people in a Charleston, South Carolina, church.

More recently, algorithms had helped lead people skeptical of vaccines to believe COVID vaccines are dangerous plots. They’ve helped convince people who didn’t like a presidential election’s outcome the election was rigged. Misinformation spread on the internet convinced a North Carolina man to drive to Washington, D.C., with his AR-15 to investigate an alleged child trafficking ring run by prominent Democrats in the basement a pizzeria. Not only was there no child trafficking ring, the restaurant has no basement. Other internet disinformation campaigns have convinced people the Sandy Hook school massacre was a staged event meant to make it easier for the government to seize guns. All the victims, according to the conspiracy theory, were “crisis actors.” No one was really hurt. According to one poll, 15 percent of Americans believe “the government, media, and financial worlds in the U.S. are controlled by a group of Satan-worshipping pedophiles who run a global child sex trafficking operation.”

Not all internet hoaxes seem to be aimed at such national targets. A New York Times Magazine article called “The Agency” tells of an explosion at a Louisiana chemical plant that never happened even though it generated official-sounding emergency alerts and social media posts complete with faked news reports – and that’s just the beginning of the story. Conspiracy theories are nothing new, of course. You’ve probably heard that Mark Twain, who died in 1910, declared, “A lie can travel halfway around the world before the truth can get its boots on.”²  

Lies seem to have picked up speed since Twain’s day. Keeping ahead of them takes work. Searching for information online means evaluating websites and individual stories. With websites, begin with the URL. If the URL ends with .lo or com.co, it’s probably not a site to trust. Sites that end with those suffixes are generally fake, sometimes mimicking legitimate sites. URLs often reveal who is behind the site, or at least what the site’s purpose is.

When evaluating a website there are several things to take into consideration, one of the first things to look at is the URL (Uniform Resource Locator: a protocol for specifying addresses on the Internet) this can often tell you several things about the website, the creator, the audience, the purpose and sometimes even the country of origin. The URL is the address you type in to get to a web site the http://uscupstate.edu/library (the University of South Carolina Upstate library’s address) or https://www.google.se/ (Sweden’s Google search).

A domain name is like a website’s proper name (the part after the www.), businesses and organizations often have a domain name that is their corporate name (for example Microsoft’s domain name is Microsoft.com).  The domain suffix is the end of the domain name (the .com part) and can offer insight into the type of organization the site is linked to.  For example, any commercial enterprise or corporation that has a web site will have a domain suffix of .com, which means it is a commercial entity. Popular domain suffixes include “.com,” “.net,” “.gov,” and “.org,” but there are dozens of domain suffixes.  However, since any entity can register domain names with these suffixes, the domain suffix does not always represent the type of website that uses the domain name. For example, many individuals and organizations register “.com” domain names for non-commercial purposes, since the “.com” domain is the most recognized.

The domain suffix might also give you a clue about the geographic origin of a web site, each country also has a unique domain suffix that is meant to be used for websites within the country. For example, Brazilian websites may use the “.br” domain suffix, Chinese websites may use the “.cn” suffix, and Australian websites may use the “.au” suffix. These country-based TLDs, sometimes referred to as “country codes,” are also used to specify different versions of an international website. For example, the German home page for Google is “www.google.de” instead of www.google.com.

Domain Suffix

This link takes you to a complete List of domain suffixes and their definitions. Here are some of the most common:

.com = Commercial site. The information provided by commercial interests is generally going to shed a positive light on the product it promotes. While this information might not necessarily be false, you might be getting only part of the picture. Remember, there’s a monetary incentive behind every commercial site in providing you with information, whether it is for good public relations or to sell you a product outright. (See the Information, Disinformation, Misinformation page)

.edu = Educational institution. Sites using this domain name are schools ranging from kindergarten to higher education. If you take a look at your school’s URL you’ll notice that it ends with the domain .edu. Information from sites within this domain must be examined very carefully. If it is from a department or research center at an educational institution, it can generally be taken as credible. However, students’ personal Web sites are not usually monitored by the school even though they are on the school’s server and use the .edu domain.

.gov = Government. If you come across a site with this domain, then you’re viewing a federal government site. All branches of the United States federal government use this domain. Information such as Census statistics, Congressional hearings, and Supreme Court rulings would be included in sites with this domain. The information is considered to be from a credible source.

.org = Traditionally a non-profit organization. Organizations such as the American Red Cross or PBS (Public Broadcasting System) use this domain suffix. Generally, the information in these types of sites is credible and unbiased, but there are examples of organizations that strongly advocate specific points of view over others, such as the National Right to Life Committee and Planned Parenthood. You want to give this domain scrutiny. Some commercial interests might be the ultimate sponsors of a site with this suffix. (See the Information, Disinformation, Misinformation page)

.mil = Military. This domain suffix is used by the various branches of the Armed Forces of the United States.

.net = Network. You might find any kind of site under this domain suffix. It acts as a catch-all for sites that don’t fit into any of the preceding domain suffixes. Information from these sites should be given careful scrutiny.

A site’s “About” tab should have a lot of information about the site, its philosophy, its staff, its mission, its owner. Those things should help you decide how credible the site is. Does the site have editors? Does the staff have experience and education relevant to their jobs? Have they worked for credible organizations? Have they worked for partisan organizations? Has the staff or the site won any awards you’ve heard of? (Promoting lots of obscure cyber awards is not a good sign. Winning lots of Pulitzer Prizes is a good sign.)

Sometimes the “About” entry is very straightforward about a site’s point of view. National Review, for instance, announces it “was founded in 1955 by William F. Buckley Jr. as a magazine of conservative opinion. The magazine has since defined the modern conservative movement and enjoys the broadest allegiance among American conservatives.”  The second word in the “About” section at The Nation is “Progressive.” The Onion  claims to be “the world’s leading news publication, offering highly acclaimed, universally revered coverage of breaking national, international, and local news events. Rising from its humble beginnings as a print newspaper in 1756, The Onion now enjoys a daily readership of 4.3 trillion and has grown into the single most powerful and influential organization in human history.” Hopefully, you recognize that’s satire.

If a site’s “About” section describes its philosophy or principles, consider not only what’s written there but how it’s written. Blaze Media, for example, says it is “news and entertainment for people who love America” while pop-up ads ask visitors to the site to give money to Blaze to “support conservative voices.” Calling itself “unapologetically pro-America and pro-free speech,” it describes the site’s opinions as “somewhere on the center-right to right of the modern American political spectrum.”

A few things there should raise at least some concern for people looking for a credible news source. While any media that responsibly exercises its First Amendment rights could rightly claim to be “pro-America and pro-free speech,” the interjection “unapologetically” suggests The Blaze thinks someone wants it to apologize for that. Couple that with its claim that it’s  “for people who love America,” implying that people who don’t read The Blaze don’t love America, and Dr. Samuel Johnson’s line about patriotism being the last refuge of a scoundrel may come to mind3. Beyond that, The Blaze identifies itself as conservative and speaks of its writers’ opinions, so it’s clearly not aiming to be an even-handed news source. The Blaze also claims to be “one of the nation’s largest and fastest-growing alternative media companies.” The Association of Alternative Newsmedia has about ninety members. The Blaze is not one of them.

The Blaze lists staff members, so readers can learn that Managing Editor Leon Wolf was previously managing editor for Red State, which describes itself as a “Conservative Blog & Conservative News Source for Right of Center Activists.” Another red flag. A site that announces it is conservative (or liberal or progressive or any other political label) is announcing it presents information from a particular point of view. Identifying with activists of any stripe is at least one step further along that biased path. ­­­­­­­­­­­­­­­­­­­­­­­­­­­­Writing from a conservative or progressive or libertarian point of view doesn’t necessarily make a site wrong, but it should make a reader more cautious about taking its information at face value.

If you’ve read what the site says about itself and you’re still convinced the site is trustworthy, or at least worth considering, look at the language and style of its headlines. Do they employ ALL CAPS? Do they use provocative or inflammatory language? Do the headlines promise something much more dramatic or scandalous than the stories turn out to be? A provocative headline can earn a lot of clicks and – since many people share stories online after reading only the headline – spread misinformation pretty effectively. Headlines are supposed to give readers a clear idea of what a story or video is about, not make them so angry or shocked they just have to share it.

When you’re ready to read the stories, see if they involve any original reporting. If a story relies primarily on reports from other media, that’s not really a news story. That’s a way of copying someone else’s work. Reputable sources will sometimes run a story that says it was first reported by another outlet, but reputable media will include their own reporting that supports or refutes that original story. If a story relies completely on reporting by another outlet, read the other outlet – after you’ve determined that one is credible, of course.

If the story includes original reporting, particularly about a controversial topic, does that reporting include quotes from people on more than one side of the issue? Does the story at least say the reporter attempted to contact the people involved?

If the story has quotes, are they in context? News sites will rarely reprint a politician’s complete speech, but be sure that the way the story has truncated the quote doesn’t cut out its original meaning. When Barrack Obama gave a campaign speech in Roanoke in 2012, he famously said, “If you’ve got a business – you didn’t build that. Somebody else made that happen.” That incensed a lot of people and his political opponents made a lot of use of it.

Here’s that line in context:

 If you were successful, somebody along the line gave you some help. There was a great teacher somewhere in your life. Somebody helped to create this unbelievable American system that we have that allowed you to thrive. Somebody invested in roads and bridges. If you’ve got a business — you didn’t build that. Somebody else made that happen. The Internet didn’t get invented on its own. Government research created the Internet so that all the companies could make money off the Internet.

The meaning of the longer quote is clearly very different than the apparent meaning of the quote out of context. Don’t let people trick you like that.

Don’t trust stories that make broad statements about large groups. “The news media,” for example, includes everything from The Salem Times-Register to The Wall Street Journal to MSNBC. Not much could possibly be true about all of them.

Even when your source has passed all those tests, if you find a news site story or a report on a university website about some research, don’t depend on that story. Check out the research itself. There’s certainly more information available in a research article than in a story about that article and it’s best to evaluate the research yourself. Harvard’s Shorenstein Center on Media, Politics and Public Policy offers a list of questions to ask about research. The first, of course, is whether the research is peer reviewed. The list includes questions about funding, sample size and the authors’ credential and potential conflicts of interest.

As elementary as it sounds, it’s important to know what you’re reading. A newspaper’s website, for example, will have several different kinds of writing. Straight news stories should just report what happened, perhaps with some context. Analysis focuses on context and meaning. Opinion pieces present the opinion of the writer. Editorials represent the opinion of the newspaper’s editorial board. If you’re reading an opinion piece that you think is straight news, you’re likely to confuse someone’s opinion with fact. To make things even more complicated, some sites allow bloggers to post under their banner, but those bloggers aren’t always held to the same standards and their work isn’t always put through a rigorous editing process.

That would be a serious mistake, but people make mistakes. That’s why it’s important to know that news sites publish corrections. If a site never admits mistakes, that certainly doesn’t mean the people who work there don’t make mistakes. It just means they don’t admit it.

Searching for reliable information on the internet is a challenge. Fiction masquerades as fact. Opinion is presented as evidence. State supported agencies seeking political gain and high school students looking to make money put are presenting fakes news as legitimate. Even with all of their trickery, one of the biggest obstacles to identifying misleading information may be you.

People are inclined to believe information that supports their views and to distrust information that challenge their views. People will even ignore information if they think it may change their minds. So, when you come across information, particularly an outrageous story, that confirms your beliefs or suspicions, try not to embrace it right away. Examine the source. Consider the evidence. Put it through all the tests described above. Then, if the story still seems solid, use the internet to verify the internet. If one news source has a big story – and it’s a real story – others are likely to have it soon. See if you can find it at other sites, other sites that have done their own reporting to verify the information, not just sites that reprinted a story or wrote a few paragraphs around a link to the original story.

Of course, you’ll have to be sure those sites are credible, too.

The following lists can help you do that.

False, Misleading, Clickbait-y, and/or Satirical “News” Sources

These tips were compiled by Melissa Zimdars, then an assistant professor of communications and media at Merrimack College, in 2016.

Tips for analyzing news sources

• Avoid websites that end in “lo” ex: Newslo (Newslo is now found at Politicops.com). These sites take pieces of accurate information and then packaging that information with other false or misleading “facts” (sometimes for the purposes of satire or comedy).
• Watch out for common news websites that end in “.com.co” as they are often fake versions of real news sources (remember: this is also the domain for Colombia!)
• Watch out if known/reputable news sites are not also reporting on the story. Sometimes lack of coverage is the result of corporate media bias and other factors, but there should typically be more than one source reporting on a topic or event.
• Odd domain names generally equal odd and rarely truthful news.
• Lack of author attribution may, but not always, signify that the news story is suspect and requires verification.
• Some news organizations are also letting bloggers post under the banner of particular news brands; however, many of these posts do not go through the same editing process (ex: BuzzFeed Community Posts, Kinja blogs, Forbes blogs).
• Check the “About Us” tab on websites or look up the website on Snopes or Wikipedia for more information about the source.
• Bad web design and use of ALL CAPS can also be a sign that the source you’re looking at should be verified and/or read in conjunction with other sources.
• If the story makes you REALLY ANGRY it’s probably a good idea to keep reading about the topic via other sources to make sure the story you read wasn’t purposefully trying to make you angry (with potentially misleading or false information) in order to generate shares and ad revenue. Thanks to Ed Brayton for this tip!
• If the website you’re reading encourages you to DOX individuals, it’s unlikely to be a legitimate source of news.
• It’s always best to read multiple sources of information to get a variety of viewpoints and media frames. Sources such as The Daily Kos, The Huffington Post, and Fox News vacillate between providing important, legitimate, problematic, and/or hyperbolic news coverage, requiring readers and viewers to verify and contextualize information with other sources.
• For more tips on analyzing the credibility and reliability of sources, please check out School Library Journal (they also provide an extensive list of media literacy resources) and the Digital Resource Center.

OpenSources Steps for Analyzing Websites

Step 1:  Title/Domain Analysis. If words like “.wordpress” or “blogger” are in the domain that usually signifies it’s a personal blog rather than a news source. If slight variations of well known websites appear, such as “.com.co,” this is usually a sign that the website is fake version of a source. However, remember that foreign reputable news organizations may have these country-specific domains.

Step 2: About Us Analysis. I usually google every title/domain name/anyone listed in the “About Us” section to see if anyone has previously reported on the website (snopes, hoax-slayer, politifact, factcheck.org, etc.) or whether it has a wikipedia page or something similar detailing its background. This is useful for identifying and correctly interpreting lesser known and/or new websites that may be on the up-and-up, such as satirical sources or websites that are explicit about their political orientation.

Then I look for information about the credentials and backgrounds of affiliated writers (is it a content mill or do they pay their writers?), editors, publishers, and domain owners (who.is etc.). It’s also useful to see if the website has a “Legal” or “Disclaimer” section. Many satirical websites disclose this information in those sections.

A total lack of About Us, Contact US, or any other type of identifying information may mean that the website is not a legitimate source of information.

Step 3: Source Analysis. Does the website mention/link to a study or source? Look up the source/study. Do you think it’s being accurately reflected and reported? Are officials being cited? Can you confirm their quotes elsewhere? Some media literacy and critical scholars call this triangulation: Verify details, facts, quotes, etc. with multiple sources.

Step 4: Writing Style Analysis. Does the website follow AP Style Guide or another style guide? Typically, lack of style guide may indicate an overall lack of editing or fact-checking process. Does it frequently use ALL CAPS in headlines and/or body text? Does the headline or body of the text use words like WOW!, SLAUGHTER!, DESTROY!? This stylistic practice and these types of hyperbolic word choices are often used to create emotional responses with readers that is avoided in more traditional styles of journalism.

Step 5:  Aesthetic Analysis. Like the style-guide, many fake and questionable news sites utilize very bad design. Usually this means screens are cluttered with text and heavy-handed photoshopping or born digital images.

Step 6: Social Media Analysis.  Look up the website on Facebook. Do the headlines and posts rely on sensational or provocative language — aka clickbait — in order to attract attention and encourage likes, clickthroughs, and shares? Do the headlines and social media descriptions match or accurately reflect the content of the linked article? (this step isn’t particularly good at helping us find fake news, but it can help us identify other misleading news sources)

How to tell good research from bad: Thirteen questions to ask

This list comes from The Journalists Resource, a product of the Harvard Kennedy School’s Shorenstein Center on Media, Politics and Public Policy. Obviously intended for journalists, it can be useful for anyone reviewing academic research. Author Denise-Marie Ordway cautions, “While there is no way to guarantee the quality of a study, these questions can help journalists avoid biased or otherwise flawed research. It’s important to note that many of these questions apply primarily to quantitative research, or research in social sciences and natural sciences that involve the analysis of data.”  You can find the complete article at this link.

1. Is this research peer reviewed? A study published in a peer-reviewed journal typically undergoes a detailed critique by a small number of qualified scholars. The peer-review process, while imperfect, is designed for quality control.

2. Is it published in a top-tier academic journal? Top journals are more likely to feature high-quality research. They are more selective about the research they accept for publication. Also, their peer-review process tends to be more rigorous.

3. Do other scholars trust this work? One indicator of whether other scholars consider a study to be credible is the number of times they cite it in their own research. It can take years, however, for a study to generate a high citation count. You can use Google Scholar. Journalists also can ask faculty in the field their opinions.

4. Who funded the research? It’s important to know who sponsored the research and what role, if any, a sponsor played in the design of the study and its implementation or in decisions about how findings would be presented to the public. Authors of studies published in academic journals are required to disclose funding sources. Studies funded by organizations such as the National Science Foundation tend to be trustworthy because the funding process itself is subject to an exhaustive peer-review process.

5. What are the authors’ credentials? Knowing where the authors work and how often they have been published can help you assess their expertise in a field of study.

6. How old is the study? In certain fields — for example, chemistry or public opinion — a study that is several years old may no longer be reliable.

7. Do the authors have a conflict of interest? Be leery of research conducted by individuals or organizations that stand to gain from the findings.

8. What’s the sample size? For studies based on samples, larger samples generally yield more accurate results than smaller samples.
9. Does the study rely on survey results? Survey results can be biased if respondents were not chosen by random selection. Beware of any survey that relies on respondents who self-select (for example, many internet-based surveys).

10. Can you follow the methodology? Scholars should explain how they approached their research questions, where they got their data and how they used it. They also should clearly define key concepts and describe the statistical methods used in their analyses. This level of detail is necessary to allow other people to check and replicate their work. Replicability is critical.

11. Is statistical data presented? Authors should present details about the data they are examining and the numerical results of their analyses. This allows others to review their calculations. In some fields, authors make their data sets publicly available.

12. Are the study’s findings supported by the data? Good researchers are very cautious in describing their conclusions – because they want to convey exactly what they learned. Sometimes, however, researchers might exaggerate or minimize their findings or there will be a discrepancy between what an author claims to have found and what the data suggests.

13. Is it a meta-study? Among the most reliable studies are meta-studies, also referred to as meta-analyses. Their conclusions are based on an analysis of multiple studies done on a particular topic.

Create a Research Strategy

Now that you know what to look for and how to evaluate what you find, how should you go about finding credible sources? Having a plan in place before you start searching will lead you to the best sources.

 Research Questions

Many students want to start searching using a broad topic or even their specific thesis statement. If you start with too broad of a topic, your search results list will overwhelm you. Imagine having to sort through thousands of sources to try to find ones to use in your paper. That’s what happens when your topic is too broad; your information will also be too broad. Starting with your thesis statement usually means you have already formed an opinion about the topic. What happens if the research doesn’t agree with your thesis? Instead of closing yourself off to one side of the story, it’s better to develop a research question that you would like the research to help you answer about your topic.

Steps for Developing a Research Question

Step 1: Pick a topic (or consider the one assigned to you).

Step 2: Write a narrower/smaller topic that is related to the first.

Step 3: List some potential questions that could logically be asked in relation to the narrow topic.

Step 4: Pick the question in which you are most interested.

Step 5: Modify that question as needed so that it is more focused.

Keywords & Search Terms

Starting with a research question helps you figure out precisely what you’re looking for. Next, you’ll need the most effective set of search terms – starting from main concepts and then identifying related terms. These keywords will become your search terms, and you’ll use them in library databases to find sources.

Identify the keywords in your research question by selecting nouns important to the meaning of your question and leaving out words that don’t help the search, such as adjectives, adverbs, prepositions and, usually, verbs. Nouns that you would use to tag your research question so you could find it later are likely to be its main concepts.

Example: How are birds affected by wind turbines?

The keywords are birds and wind turbines. Avoid terms like affect and effect as search terms, even when you’re looking for studies that report effects or effectiveness. These terms are common and contain many synonyms, so including them as search terms can limit your results.

Example: What lesson plans are available for teaching fractions?

The keywords are lesson plans and fractions. Stick to what’s necessary. For instance, don’t include: chil- dren—nothing in the research question suggests the lesson plans are for children; teaching—teaching isn’t necessary because lesson plans imply teaching; available—available is not necessary.

Keywords can improve your searching in all different kinds of databases and search engines. Try using keywords instead of entire sentences when you search Google and see how your search results improve.

For each keyword, list alternative terms, including synonyms, singular and plural forms of the words, and words that have other associations with the main concept. Sometimes synonyms, plurals, and sin- gulars aren’t enough. Also consider associations with other words and concepts. For instance, it might help, when looking for information on the common cold, to include the term virus—because a type of virus causes the common cold.

Once you have keywords and alternate terms, you are prepared to start searching for sources in library search engines called databases.

Finding Sources

The college library subscribes to databases (search engines) for credible, academic sources. Some are general purpose databases that include the most prominent journals in many disciplines, and some are specific to a particular discipline. Brown Library’s website includes a database list containing over one hundred search engines, organized by subject area.

Sometimes the online database list is overwhelming for students. Please remember, you can always seek advice from librarians on the best databases for your topic. Librarians have also created InfoGuides which contain a list of databases and other credible resources for different programs and  courses  offered  at  VWCC. Your  English  111  InfoGuide is the best place to start your research for this class. View the “Finding Articles” tab to see a list of databases that may work for your topic.

__________________________________________________________________________

1People have been around for about 6,666 generations because homo sapiens appeared about 200,000 years ago and a generation is about 30 years. An internet search turned up more than 5.7 million answers. The first one estimated 87 to 103 generations. The “generation” entry at the top of the first page was about a 1955 Polish movie.

2You’ve probably heard that, but the first time it was credited to Twain was nine years after he died. It seems to have evolved from something Jonathan Swift, the author of “A Modest Proposal,” wrote in 1710. Twain, along with Winston Churchill and Oscar Wilde, get credit for all sorts of clever sayings, some of which they actually said.

3 According to a site called Quote Investigator (which passed a credibility test), “biographer James Boswell credited Samuel Johnson with the adage within a diary entry dated April 7, 1775. Different versions have evolved during the ensuing decades and centuries.” Johnson gets credit for all sorts of sayings, some of which he said, so it’s best to check these things out.

“Evaluating Sources” uses information from:

Informed Arguments: A Guide to Writing and Research  by Terri Pantuso, Sarah LeMire, and Kathy Anders, which was published in 2019 by Texas A&M University with an attribution, non-commerical, share-alike creative commons license. It was accessed through the Open Textbook Library at this url: https://open.umn.edu/opentextbooks/textbooks/informed-arguments-a-guide-to-writing-and-research-pantuso.

Let’s Get Writing by Elizabeth Browning, Kirsten DeVries, Kathy Boylan, Jenifer Kurtz, and Katelyn Burton, which was published by Virginia Western Community Colleges and uses a Creative Commons Attribution 4.0 International License,