Surveys | Dynamic Network Lab | Teachers College, Columbia University

Information for Researchers about the Lab's Methodological Tools

Overview

Our Lab's Surveys, Calculators, App, and Programming examine various decision making, reasoning, and social network processes. The tools are theoretically grounded in Behavioral Reasoning Theory (BRT) and Dynamic Network Theory (DNT). Our Lab also created network goal analysis (NGA) as a method that corresponds directly with DNT. Researchers can apply our decision and network tools to numerous contexts, such as examining goals, decisions, behaviors, or networks in organizational, health, medical, family, marketing, clinical/counseling, well-being, relationship, or performance contexts.

THE DECISION-MAKING CALCULATOR BASED ON BRT:

Decision-making calculator preview (This shows a person hypothetically responding to the decision to return to work in person)

This survey is based on BRT's concepts. The calculator aims to help people with important decisions in their lives and provides users with live score feedback. For researchers, BRT surveys below can be targeted on specific decisions or behaviors of interest, such as job turnover, health/exercise behaviors, family decisions, or purchasing behavior, as further detailed in the next section. Click here for more theoretical information about BRT. If you or other people you know want to use the calculator in their real life decision making, click here to go to the Live Decision Making Calculator Page.

--Following Processes Coming soon-- (DRAFT)

METHODOLOGICAL TOOLS FOR RESEARCHERS*

1. Behavioral Reasoning Theory (BRT)

Click below to see an example of a behavioral reasoning theory survey applied to employee turnover (without calculator dials)

Original BRT survey preview. Assessed: reasons for, reasons against, global motives (attitude, subjective norm, and perceived control), and intention. When possible, researchers should aim to examine a follow-up behavior via self-report questions or observation at least one month after the initial assessment to further test the theory over time, which helps mitigate common-method bias (optional). Extra pages for demographics, etc., included.
- With the BRT survey file open in the Qualtrics program, a researcher can input the decision/behavior they are interested in studying (e.g., "stay at my job" for a retention study) by simply entering it into the file in the "defaults option" for the item (one click to confirm your entry and the survey is contextualized to your specific decision or behavior of interest).
New BRT survey preview: Above items plus additional reason comparison and counterargument scales (forthcoming)
As is normal with Qualtrics, the CSV output from Qualtrics would need to be transformed into relevant quantitative variables in the dataset (e.g., averages across each global motive, such as attitude, and intention - or used in CFA or SEM) and reasons (e.g., summing across the rated reasons for and against the given behavior). Although the above processes may be straightforward for seasoned researchers, additional consultation on how to transform CSV variables from Qualtrics output into SPSS sysntax or R code can be potentially ascertained by contacting the Lab (availability restrictions may apply).

2. DNT Network Goal Analysis and App

Click below to see a full example visualization and statistic report of a hypothetical team trying to achieve a mission in a competitive context. This was created through our NGA app / Dynamic network app (based in R and R Shiny) after inputting CSV results from the network goal survey from Qualtrics that collected.

Example Network Goal Analysis Report for Participant/Analyst responding to questions about their network to "Achieve Our Mission":

NGA Visualization file - Dynamic HTML

DETAILS:

The following steps provide more details about how the above report was produced:

Step 1. Use our "Open-Ended Network Goal Analysis Survey".

Network goal analysis (NGA) preview.

This survey allows researchers / users to insert any goal (or behavior) of interest. Up to 30 actors in a social network can be analyzed around the focal goal (revision in process).
This is the full survey version which comprehensively tests DNT variables (Westaby, 2012; Westaby & Parr, 2020; Westaby, Pfaff, & Redding, 2014), such as all the motivational social network roles (G, P, S, V, N, R), system competences, and many performance DV measures are included, which are not required for visualization and could be omitted - Additional survey version possible without extra measures.
Researchers would populate the survey via participants under study or a research analysist examining a specific system (or multiple systems). This survey becomes longer as the number of entities increase; hence, it's recommended for participants/analysts to take breaks when researchers desire to examine the full richness of a participant's or analyst’s focal system that has many Actors. Less entities take considerably less time. Our philosophy is that if researchers are interested in truly examining complex systems, it sometimes takes more time to do so; fortunately, using our advanced surveys makes this reasonably doable as long as participants are aware of this and its importance for understanding complex systems.
See "Specific Goal Survey" or "Case Study Survey" tabs at the bottom of this page for more details about how participants can set-up the goal and actors for a network goal analysis. Also, if interested, see Network goal analysis paper (Westaby & Parr, 2020) for the analysis of single goal systems in complex social networks using the above tools - one is for a team goal, the other is for an individual goal; more research is needed on larger samples with mulltiple goals.

Step 2. Have Qualtrics program download the survey results into a CSV file. Basic familiarity with Qualtrics is helpful.

Example CSV output file from the Qualtrics survey in the simple example above:

CSV needed to activate NGA visualization

Remember, this survey assesses up to 30 possible actors across major roles in DNT (although usually far fewer actors are typically examined) so there are thousands of variables in the actually CSV file to account for all relevant links. However, the assessed actors and their links are easily visualized in the App once data is collected. Another advantage: Because the advanced Qualtrics survey only responds (pipes) to the actors entered into the network, the survey is relevant to a participant's own system of potential relations and the CSV file typically uses a much smaller set of the potentially thousands of variables in the file, which are simply there to account for all potential dyadic relations that could exist in a system with up to 30 actors. If/when you examine the example CSV file for this relatively small system example above, you will see that only a small sub-set of the variables have data to capture the important network behaviors visualized in the Network Goal App / Dynamic Network App.

Step 3. Open the Lab's Network Goal Analysis App (also known as the Dynamic Network App). Click below to see an earlier draft version of the App, but remember that you will need to upload the CSV file from Step 2 into the App to see results. You also need to save that CSV file onto your computer for the upload, as clarified more below. So, click on the following link, check it out briefly in its blank version, and then upload the CSV file you saved on your computer into the App to see detailed network dynamics and statistics on the App itself. For researchers collecting their own data, you would be inserting your own CSV files from your Qualtrics survey results.

https://dntvisuals.shinyapps.io/DNTApp_4_19_2019/

Step 4. Upload the Qualtrics CSV file from the survey into the App at "Upload here" and click "Activate". Then the visualizations will populate and appear in the App. BUT even more useful is using the "Download report" button which orders all the data output you need from the App in a systematic fashion, based on DNT, so you can more easily understand each system and sub-system according to DNT and unpack complex systems in a straightforward manner. In addition, because the report can be saved as a file, you have a permanent record of your visual and statistical results without needing to search for it in the App and you do not need to take screen shots for your records. Most major results are shown in the report for researchers. To help our Lab in its efforts to freely provide these resources to researchers, please use your downloaded report more than looking at the same results on the App itself, because it saves on the Lab's processing costs for R shiny. We provide this processing as a contribution to the network science community for free (as long as our budget permits).

To note, when uploading your own data file, your file must have only 1 row of data to capture a single system for analysis in the App's visualizaitons. Hence, if you have multiple participants/analysts examining separate systems, please use separate CSV files for each one and run them separately. (Keep the top 3 rows from the Qualtrics CSV as well, such as variable name and description - do not delete them - since they are accounted for in our coding). The output file from R shiny will provide a full analysis for each of the systems that you run separately. If you want between-system metrics or summaries, you will need to extract the data from each report and compile into an additional datafile, which would be straightforward. A goal for future research, method, and code is to automate this process. Computer scientists are encourgaged to reach out to our Lab if interested in helping with these missions.

Step 5: Enjoy examining and learning about the richness of the network dynamics involved in the focal goal, especially from the report. Because the App and its downloadable report are formatted in HTML, you can move actors and networks around by clicking on them and dragging them to new locations in a dynamic fashion, which goes beyond static reports. Remember, having the downloaded report will be a record of each network analyzed and there is no need to rerun the App (or keep it open), once you download your report for each system measured. We hope this is helpful to researchers.

Summary

What is the App doing technically? It translates the Qualtrics CSV file into Visual Network Goal Analysis and allows you to download a full NGA report in a user-friendly HTML file for each of the systems and sub-systems examined.

Survey and App updating / Coding Improvements

Programming and Content Engineering Leads on current method and App versions: Adam K. Parr and James D. Westaby respectively (alphabetical) with various support from internal and external colleagues. When required for scientific publications, the R code can be requested by contacting the Lab for potential approval. The Lab welcomes recommendations for and mutually sharing of App improvements in code and process, in the spirit of open-source programming promoted in the R statistics program (and many leading scientists) and we look forward to including improvements from the scientific community in future updates to the App. Please check back on our Lab website for formal updates and approval processes. The most recent App formally approved by the Dynamic Network Lab will be available on this website. Beta versions and updates from additional collaborators will be denoted as such.

Feel free to contact the Lab if you are interested in collaborating with the Lab on specific applications (availability restrictions may apply). It's often helpful if their is potential funding available to ease potential additional programming/processing needs; proposing ideas for securing grants in partnership is also possible.

DOWNLOADING OUR LAB'S QUALTRICS SURVEYS AND PROGRAMMING* Researchers can email Professor Westaby to request potential permission to use our Qualtrics Surveys for scientific research. Please indicate your name, your university/research institution, basic purpose of the study and confirmation that you are/will be obtaining IRB approval and abide by APA guidelines. People requiring R code for our App, based on required scientific justification, also need to have familiarity with R program and its philosophy for open-source programming, and we request sharing updates with the Lab so that all advances can be located in one convenient location for users.

** As is the case for the popular R programming, our surveys, calculators, Apps, and outputs are not guaranteed, given similar arguments made in R programming. Moreover, researchers' implemention of processes are often beyond the Lab's control. The Lab requests that researchers share any anomalies or bugs found in our surveys or programming, so that improvement can be passed on to the scientific community.

If general actor categories are desired (instead of individual actors such as discussed in the "Specific Goal Survey", network goal surveys can be populated with such generalization in the following general way to examine their perceived impact on participants' major life activities. Technically, this approach examines the participant’s important general goal of “getting their important things done” and then the pre-determined categories below (or more granular as needed) are inserted as actors to keep the survey relatively short and relevant. Participants would only use the general categories below that are relevant in their lives. Framing the survey in the above manner allows it to generalize across the numerous activities (and goals) we are engaged in during our daily life pursuits. It's a novel way to abstract up to a higher level, yet grounded in concrete measures and visualizations. In DNT, it uses the "entity abstraction" process (Westaby, 2012), which is a common practice people use to describe their complex systems.

Actor Categories Commonly Used in the Life Network Survey

You (Focal Entity)
Spouse/Partner
Children
Other family members
Friends
People at work
Neighbors
Other important people

Additional or other categorical formulatioins are possible, or researchers can use the specific goal survey to specify up to 30 individual actors at the current time.

Why Theory Matters

Dynamic network theory shows how the actors above, for example, are perceived to impact each other in relation to the general goal of getting important things done. Thus, it's a unique way to show how critical entity categories are connected in our most important activities. Dynamic network theory explains how this all happens with the data collected.

How Long Does It Take?

This type of survey with abstracted entity names is relatively shorter, since a shorter list of pre-determined entity categories are used, based on common entities used in daily life. It usually can be done in about 30-45 minutes (with only main DVs), although researchers should encourage participants to take breaks as needed.

This survey examines an employee's common goal of “getting their important things done at work”. The pre-determined categories below can help keep the survey relatively short and relevant. Participants would only use the general categories below that are very common at work. No last names are collected in the survey. Framing the survey in the this manner allows it to generalize across the numerous activities (and goals) employees are engaged in at work. It's also a novel way to abstract up to the highest level, yet grounded in concrete measures and visualizations.

Actor Categories Commonly Used in the Work Network Survey

You
Coworkers
Management
Family
Friends
Other contacts

Additional refinement possible, such as management being broken down into supervisor, executive managemment, and human resources.

Why Theory Matters

Dynamic network theory shows how the entities above, for example, are perceived to impact each other in relation to the general goal of getting important things done at work. Thus, it's a unique way to show how our critical entity categories are connected in our most important work activities. Dynamic network theory explains how this all happens with the data collected.

How Long Does It Take?

This is a relatively shorter survey, since a short list of pre-determined categories are used, based on common categories used in life. It usually can be done in less than 30 minutes, although we also encourage participants to take breaks as needed.

This survey can assess any goal or behavior a participant is interested in pursuing. Example goals people have studied in the past: doing a project, getting a job, losing weight, quitting tobacco, exercising more, etc. Hence, some users like this survey because it lets them input their own goal and entity names, although users are asked not to use last names in surveys in order to maintain privacy and confidentiality of participants and their networks.

This survey is also helpful when people want to achieve an important goal and want to intervene in their system to create change. Researchers can examine this process. For example, in the Social Networks and Performance course that partners with the Dynamic Network Lab, participants do this survey at the beginning of the semester, create a network goal intervention to help achieve their goal, and do another survey at the end of the semester to see how well they have done. Anecdotally, we have seen many people achieve goals in their lives through our process.

Preparing for the survey

We encourage participants to think carefully about the important goal they decide to examine. They should also carefully consider the people involved with it, before starting the survey, including people that may have positive or negative influences. They can see the helpful tips below before starting the survey. Our tools allow participants to complete the survey on a computer or smartphone.

Tips

Which goal or behavior should I examine?

It can be anything that is meaningful or important to participants. It can be broad or specific. At a broad level, it could represent "Getting things done". Or at a specific level, it could represent "Doing X", "Getting X", "Pursuing X", etc. Some popular examples: Getting a job, finishing my project, losing weight, starting a business, eating healthy, getting into a university, finding the right person, exercising more, taking a class, getting a promotion (or pay raise), stopping discrimination (or racism, sexism, ageism, etc.), getting involved in my community, buying X, taking care of my kids, making x, reducing a major conflict, making more money, reducing (or quitting) drinking or drugs, reducing high stress, volunteering more, etc. Additionally, leaders could examine much larger organizational or societal goals that they are trying to advance in their systems. Participants can feel free to focus on what serves them best and is important in their lives or others' lives.

Who is specifically in my network?

Participants would be listing important people involved with their goal, both those generating positive and/or negative influences. Participants could use first names, initials, or even secret code names in the survey; alternatively, they can also use broader (and natural) category names that they normally use to describe those involved in their goal pursuits. This can also help keep the network goal survey shorter (often under 40 min) and fitting with how they portray the system in the real world (e.g., My "friends" help on project X), since the survey gets longer as participants add more individual actors to the assessment. The survey becomes longer in larger systems because the survey assesses all combinations of linkages to properly assess the full network system, grounded in DNT. It is perfectly fine to have a long and properly detailed actor list, but participants need to be prepared to take more breaks filling out the survey, so they are not fatigued, which helps promote quality data.

Example general categories: The following categories commonly appear among participants using the survey, and participants are free to use them, if they make sense to their goal pursuit: You, significant other, friends, family, coworkers (and others they would insert that play a unique role beyond these). The following examples are also common for understanding many work goals: You, coworkers, supervisor, management, family, friends (and others they would insert that play a unique role beyond these). Again, if the above examples don't make the best sense for participants and their goal pursuit, they should be encouraged to freely use their own terms that make the most sense for this version of the survey. Participants should take time to refine and confirm their actor list before starting, to ensure it's stable and not needing more changes.

How long does this survey take?

This survey can be short or long, depending on how many actors participants list in the survey. This can dramatically range from several minutes, in the case of just a few entities to well over an hour if dozens of entities are assessed. Participants can also enter up to 30 actors, although the time it takes to complete the survey with this many entities increases even further. However, this can be justified in important cases and would often require simply taking more frequent breaks to complete the system assessment. The results will likely portray a rich pattern of relations for analysts that can be broken down into meaningful chunks. Click here to review visualizations.

Why Theory Matters

Dynamic network theory shows how the actors participants indicate are perceived to impact each other in relation to the selected goal. Dynamic network theory explains how this happens.

The case study survey allows researchers or interested analysts to examine how a complex network is involved in an important goal or behavioral case, typically of major import. The computer-generated results allow researchers to dynamically visualize the network results online. This can help promote insight into key motivational and behavioral dynamics involved in the given case.

These case studies can focus on an individual, group, organization, or society that is having major impact on others. We typically recommend focusing the survey on the main Focal Actor and its most critical goal (or behavior) that is impacting the larger system to keep the analysis nicely focused. This is a useful ego-centric analysis that keeps nice boundaries. By focusing on the Focal Actor and its goal, researchers can see how others in the network are supporting or resisting this Actor in many ways.

Although researchers could also examine the multiple goals of different Focal Entities in a system (such as doing a separate survey for each goal, we encourage analysts to find one of the most important goals being pursued in the system that is having a large positive or negative effect on others as a start. A practical way to do this is for researchers to make sure that they select a goal that most others (or experts) would recognize as being critically impactful. In this way, the results should relate to audiences that are seriously interested in the results of the case study examination.

The following examples illustrate various cases that could be examined with our case survey tool. The Focal Actor and the focal goal are shown.

Societally Impactful Examples

Martin Luther King's pursuit of civil rights
Steve Jobs starting Apple
Gandhi's non-violent pursuit of India's freedom
N. Korea pursuing nuclear weapons
U.S. entering WWII
U.S. sending astronauts to the moon

Other General Examples (Analysts would fill in X and Y as relevant to their cases)

Organization X pursuing mission Y
Entity X pursuing goal Y
Employee X trying to do Y
Person X doing Y

Such analyses could examine cases that are presumed to have positive outcomes (e.g., those that have inspired others) or negative outcomes (e.g., those involving unethical or illegal actions that harm others).

More General Goals

The network goal surveys and analytics can also be focused on broader group or organizational goals in a non-ego-centric manner. This provides a different analytical lens, but important for some case studies, such as examining how various entities are pursing and supporting a team's goal of interest.

How Long Does This Survey Take?

Like the specific goal survey, the case survey can be short or long, depending on how many entities are listed in the case. The time can dramatically range from just several minutes, in the case of only a few entities to well over an hour if dozens of entities are assessed. Participants can enter up to 30 actors for a single goal, although the time it takes to complete the survey with this many entities increases even further. This is often justified in important cases and typically requires taking more frequent breaks to complete the system assessment. This mitigates error in assessment. The result will likely be a stunning portrayal of the full complexities, but can be broken down by blocks to understand the motivational dynamics. Click here for for more info. about visualization.

There are two primary ways indivduals participate in the methodological tools from the Lab:

(1) external researchers run studies with participants, which utilize our surveys and/or processes and run analyses using our various Lab tools or

(2) participants are directly involved with active research projects or processes being conducted by researchers in the Dynamic Network lab, such as individuals who are enrolled in the following courses, workshops, or activities in the Columbia University Community, some of which are for university credit.

Current studies in organizations or large survey samples
Social networks and performance course (M.A. level). Participants execute network goal interventions on an important goal in their own lives, using Lab tools.
Dynamic networks and systems course (Ph.D. level seminar).
Using survey research in organizational consulting course (M.A. level).
Dynamic network lab (typically Ph.D. level).
Proseminar in social psychology course (Ph.D. level).
Dissertations (Ph.D. level)

Space and availability restrictions may apply.

Comprehensive Assessments

NETWORKS AND GOAL PURSUIT

Our network goal surveys are scientifically designed for people that are serious about taking the time needed to assess network dynamics more fully. The bigger the system, the more time it will take to assess, which we strongly believe is fitting in such systems. This is because our visualizations and analyses can help break down the complexity to understand important motivational elements. Less comprehensive survey approaches, such as measuring only one type of network linkages variables (e.g., support), could under-estimate or miss what is really happening in the system and thus limit the ability to create effective change (e.g., missing conflict linkages thay may powerfully impact people). Some of our surveys are shorter than others. For example, the life network and work network surveys often use a smaller set of common pre-determined entity categories, which is helpful to make meaningful comparisons across participants.

Scientifically, our predictor variables from dynamic network theory can be examined for validity on performance scales within and between systems (e.g., using regression and multi-level analyses). Performance scales are embedded within each survey. Hence, our network approach is quite unique in its capacity to use theoretical variable to not only describe relations in dynamic network systems, but also predict emergent outcomes, such as macro performance, achievement, climates, and/or system well-being.

INDIVIDUAL DECISION-MAKING

Our Decision-Making Calculator assess the reasoning processes involved in people making important decisions as well as trying to better understand and positively influence decision confidence, satisfaction, and mitigate regret.

Decision-Making Calculator

PAST HEALTH CALCULATORS FOR COVID-19 PREVENTION

This calculator, which was used during the first major contagion in the U.S., provided a safety score and individuals received personalized feedback about their prevention preparedness right inside the calculator. It was anonymous, free, and short (usually taking less than 10 min). It was based on the CDC recommendations with elements of Behavioral Reasoning Theory measured. Reason variables were shown to demonstrate predictive validity.

The research aspect of the calculator had been aimed at providing important information for emergency prevention guidelines to help fight COVID-19 (TC IRB #20-260)

Group and Team Analysis

Group and team dynamics can also be assessed through our network goal surveys or observation, given that most teams, for example, have goals and missions. There are different ways researchers can collect data and examine such dynamics:

Techniques

Shared analysis. A group goal can be indicated and all team members participate in the survey to share and assess their views about the network dynamics. Such an approach could be utilized in team development and action learning.
Representatives. If a representative, such as a team leader or manager, wants to assess the team, the leader can complete a survey about the leader's perceptions of the dynamic network system. Such data can be confidentially used by the leader to help create positive change in the system. We often recommend that at least two representatives engage in the survey (either together or separately) so that reliability can be assessed. See forthcoming publications in our lab for specific techniques to assess reliability among multiple representatives.
Each team member. Each team member can participate in a survey concerning each member's work and then the computer visualization and analysis connects all members to show the full system of motivational factors around everyone’s major goals. We recommend using the specific goal survey applied to the goal of “getting your important things done on the team”. This approach also allows for the analysis of dynamic network intelligence for each member and the group overall.
Observational method. Instead of self-report surveys, the Lab is working on new methods to assess live group decision-making by joining the decision-making calculator and network goala analysis to show how people in a group are discussing various topics (G) and supporting one another (S) or not (P, V, and N) as well as constructive relations in the process.

When multiple groups or teams are assessed, there are ways to analyze them separately and as part of the entire given organization, thereby taking a multi-level perspective, but grounded in the dynamic network theory perspective around goals. Additional programming would be needed.

More Information

How does the survey process work?

Learn more

What does the computerized report look like?

Learn more

Click to sign-up for those eligible to participate

Click Here