5.2 Finding and Evaluating Research Sources

Magazines like Psychology Today are popular sources. Academic journals like the Journal of Marriage and Family are scholarly
Figure 5.2.1 Examples of Popular vs Scholarly Sources (Last, 2019).

In this “information age” when so much information is available at our fingertips on the Internet, it is crucial to be able to critically search through the reams of information in order to select credible sources that can provide reliable and useful data to support your ideas and convince your audience. In the era of “fake news,” deliberate misinformation, and “alternative facts,” developing the skill to evaluate the credibility of sources is critical.

Since you have completed COM101 and COM111, you are familiar with academic journals and how they differ from popular sources, as in Figure 5.2.1. These contain peer-reviewed articles written by scholars, often presenting their original research, reviewing the original research of others, or performing a “meta-analysis” (an analysis of multiple studies that analyze a given topic). If you would like to refresh your memory on this, watch this Seneca Libraries video: Popular and Scholarly Resources.

 

 

Scholarly articles published in academic journals are usually required sources in academic research essays; they are also an integral part of engineering projects and technical reports. Many technical projects require a literature review, which collects, summarizes and sometimes evaluates the work of researchers in a field whose work has been recognized as a valuable contribution to the “state of the art.” However, they are not the only kind of research you will find useful. Since you are researching in a professional field and preparing for the workplace, you will draw upon many credible kinds of sources in a professional context.

Table 5.2.1 lists several types of sources you may find useful in researching your projects. You should also consult Seneca Libraries Writing and Communicating Technical Information: Key Resources, where you will find key databases for technical research.

Table 5.2.1 Typical research sources for technical projects
[Skip Table]
Source Type Description
Academic Journals, Conference Papers, Dissertations, etc.

Scholarly (peer-reviewed) academic sources publish primary research done by professional researchers and scholars in specialized fields, as well as reviews of that research by other specialists in the same field.

For example, the Journal of Computer and System Sciences publishes original research papers in computer science and related subjects in system science; the International Journal of Robotics and Animation is one of the most highly ranked journals in the field.

Reference Works

Specialized encyclopedias, handbooks, and dictionaries can provide useful terminology and background information.

For example, the Kirk-Othmer Encyclopedia of Chemical Technology is a widely recognized authoritative source.

You may cite Wikipedia or dictionary.com in a technical report, but be sure to compare the information to other reliable sources before use.

Books

Chapters in Books

Books written by specialists in a given field usually contain a References section that can be very helpful in providing in-depth context for your ideas.

For example, Designing Engineers by Susan McCahan et al. has an excellent chapter on effective teamwork

Trade Magazines and Popular Science Magazines

Reputable trade magazines contain articles relating to current issues and innovations; therefore, they can be useful in determining what is “cutting edge,” or finding out what current issues or controversies are affecting the industry.

Examples include Computerworld, Wired, and Popular Mechanics.

Newspapers (Journalism)

Newspaper articles and media releases offer a sense of what journalists and people in industry think the general public should know about a given topic. Journalists report on current events and recent innovations; more in-depth “investigative journalism” explores a current issue in greater detail. Newspapers also contain editorial sections that provide personal opinions on these events and issues.

Choose well-known, reputable newspapers such as The New York Times.

Industry Websites (.com)

Commercial websites are generally intended to “sell,” so you have to select information carefully, but these websites can give you insights into a company’s “mission statement,” organization, strategic plan, current or planned projects, archived information, white papers, technical reports, product details, costs estimates, etc.

Organization Websites (.org)

A vast array of .org sites can be very helpful in supplying data and information. These are often public service sites and are designed to share information with the public.

Government Publications and Public Sector Websites (.gov/.edu/.ca)

Government departments often publish reports and other documents that can be very helpful in determining public policy, regulations, and guidelines that should be followed.

Statistics Canada, for example, publishes a wide range of data.

University websites also offer a wide array of non-academic information, such as strategic plans, facilities information, etc.

Patents

You may have to distinguish your innovative idea from previously patented ideas; you can look these up and get detailed information on patented or patent-pending ideas.

Public Presentations

Public consultation meetings and representatives from industry and government speak to various audiences about current issues and proposed projects. These can be live presentations or video presentations available on YouTube or TED talks.

Other

Can you think of some more? (Radio programs, podcasts, social media, etc.)

The importance of critically evaluating your sources for authority, relevance, timeliness, and credibility cannot be overstated. Anyone can put anything on the internet; and people with strong web and document design skills can make this information look very professional and credible—even if it isn’t. Since much research is currently done online, and many sources are available electronically, developing your critical evaluation skills is crucial to finding valid, credible evidence to support and develop your ideas. In fact, this has become such a challenging issue that there are sites like this List of Predatory Journals that regularly update its online list of journals that subvert the peer review process and simply publish for profit.

 

Knowledge Check

 

Mark Twain, supposedly quoting British Prime Minister Benjamin Disraeli, famously said, “There are three kinds of lies: lies, damned lies, and statistics.” On the other hand, H.G. Wells has been (mis)quoted as stating, “statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write” (Quora, n.d.). The fact that the actual sources of both of these “quotations” are unverifiable makes their sentiments no less true. The effective use of statistics can play a critical role in influencing public opinion as well as persuading in the workplace. However, as the fame of the first quotation indicates, statistics can be used to mislead rather than accurately inform—whether intentionally or unintentionally.

When evaluating research sources, be careful to critically evaluate the authority, content, and purpose of the material, using questions in Table 5.2.2. You may also want to view the brief Seneca Libraries video, Evaluating Websites.

 

 

Table 5.2.2 Evaluate the authority, content, and purpose of information
[Skip Table]
Authority
Researchers
Authors
Creators

Who are the researchers/authors/creators? Who is their intended audience?

What are their credentials/qualifications? What else has this author written?

Is this research funded? By whom? Who benefits?

Who has intellectual ownership of this idea? How do I cite it?

Where is this source published? What kind of publication is it?

Authoritative Sources: written by experts for a specialized audience, published in peer-reviewed journals or by respected publishers, and containing well-supported, evidence-based arguments.

Popular Sources: written for a general (or possibly niche) public audience, often in an informal or journalistic style, published in newspapers, magazines, and websites with a purpose of entertaining or promoting a product; evidence is often “soft” rather than hard.

Content

Methodology

What is the methodology of their study? Or how has evidence been collected?

Is the methodology sound? Can you find obvious flaws?

What is its scope? Does it apply to your project? How?

How recent and relevant is it? What is the publication date or last update?

Data

Is there sufficient data here to support their claims or hypotheses?

Do they offer quantitative and/or qualitative data?

Are visual representations of the data misleading or distorted in some way?

Purpose
Intended Use and Intended Audience

Why has this author presented this information to this audience?

Why am I using this source?

Will using this source bolster my credibility or undermine it?

Am I “cherry-picking” –  using inadequate or unrepresentative data that only supports my position, while ignoring substantial amount of data that contradicts it?

Could “cognitive bias” be at work here? Have I only consulted the kinds of sources I know will support my idea? Have I failed to consider alternative kinds of sources?

Am I representing the data I have collected accurately?

Are the data statistically relevant or significant?

Given the pie chart in Figure 5.2.2, if you only consulted articles that rejected global warming in a project related to that topic, you would be guilty of cherry-picking and cognitive bias.

Pie chart showing that of the 13,950 peer-reviewed climate articles between 1991-2012, only 24 rejected global warming
Figure 5.2.2 The number of articles that reject global warming out of all peer-reviewed climate articles within a 21 year time period (DeSmog Blog, n.d.).

Beware of Logical Fallacies

We all have biases when we write or argue; however, when evaluating sources, you want to be on the lookout for bias that is unfair, one-sided, or slanted. Consider whether the author has acknowledged and addressed opposing ideas, potential gaps in the research, or limits of the data. Look at the kind of language the author uses: is it slanted, strongly connotative, or emotionally manipulative? Is the supporting evidence presented logically, credibly, and ethically? Has the author cherry-picked or misrepresented sources or ideas? Does the author rely heavily on emotional appeal? There are many logical fallacies that both writers and readers can fall prey to (see Table 5.2.3). It is important to use data ethically and accurately, and to apply logic correctly and validly to support your ideas.

Table 5.2.3 Common logical fallacies
[Skip Table]
Bandwagon Fallacy

Argument from popularity – “Everyone else is doing it, so we should too!”

Hasty Generalization

Using insufficient data to come to a general conclusion.

E.g., An Australian stole my wallet; therefore, all Australians are thieves!

Unrepresentative Sample

Using data from a particular subset and generalizing to a larger set that may not share similar characteristics.

E.g.,  giving a survey to only female students under 20 and generalizing results to all students.

False Dilemma

“Either/or fallacy” – presenting only two options when there are usually more possibilities to consider

E.g.,  You’re either with us or against us.

Slippery Slope

Claiming that a single cause will lead, eventually, to exaggerated catastrophic results.

Slanted Language

Using language loaded with emotional appeal and either positive or negative connotation to manipulate the reader

False Analogy

Comparing your idea to another that is familiar to the audience but which may not have sufficient similarity to make an accurate comparison

E.g., Governing a country is like running a business.

Post hoc, ergo prompter hoc

“After this; therefore, because of this”

E.g., A happened, then B happened; therefore, A caused B.

Just because one thing happened first, does not necessarily mean that the first thing caused the second thing.

Circular Reasoning

Circular argument – assuming the truth of the conclusion by its premises.

E.g.,  I never lie; therefore, I must be telling the truth.

Ad hominem

An attack on the person making an argument does not really invalidate that person’s argument. It might make them seem a bit less credible, but it does not dismantle the actual argument or invalidate the data.

Straw Man Argument

Restating the opposing idea in an inaccurately absurd or simplistic manner to more easily refute or undermine it.

Others?

There are many more… can you think of some?

For a bit of fun, check out Spurious Correlations.

 

Knowledge Check

 

Critical thinking lies at the heart of evaluating sources. You want to be rigorous in your selection of evidence because, once you use it in your paper, it will either bolster your own credibility or undermine it.

Notes
  1. Cover images from journals are used to illustrate the difference between popular and scholarly journals, and are for noncommercial, educational use only.
References

Government of Canada. Statistics Canada. http://www.statcan.gc.ca/eng/start

Last, S. (2019). Technical writing essentials. BCcampus. https://pressbooks.bccampus.ca/technicalwriting/

Seneca Libraries. (2013, July 2). Evaluating Websites [Video]. YouTube. https://youtu.be/35PBCC5TKxs

Seneca Libraries. (2013, July 2). Popular and Scholarly Resources [Video]. YouTube. https://youtu.be/wPj-BBB0le4

Seneca Libraries. (2021, January 4 updated). Writing and Communicating Technical Information: Key Resources. Seneca College. https://library.senecacollege.ca/technical/keyresources

What is the source of the H.G. Wells quote, ‘Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write/”? (n.d.). Quora.  https://www.quora.com/What-is-the-source-of-the-H-G-Wells-quote-Statistical-thinking-will-one-day-be-as-necessary-for-efficient-citizenship-as-the-ability-to-read-and-write

Why climate deniers have no credibility — in one pie chart.  DeSmog Blog. https://www.desmogblog.com/2012/11/15/why-climate-deniers-have-no-credibility-science-one-pie-chart

License

Icon for the Creative Commons Attribution 4.0 International License

Technical Writing Essentials Copyright © 2019 by Suzan Last is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book