Hot answers tagged

6

It's important to distinguish between measures and analyses, because only analyses can be quantitative or qualitative, not measures. Measures are, essentially, systematic processes by which we acquire our data, and analyses are processes we use to look at the data. As a rule of thumb, the difference is not hard to find and is given in the name: ...


6

ANOVA and t-tests are statistical tests for significance and therefore quantitative. The other mentioned items are scales (adding numbers to a certain choice) and therefore they can be considered as ordinal scales, and hence as quantitative as they are based on numbers. The NASA one can be administered by using a sliding scale which can be considered to ...


5

It depends on the types of changes you are looking to make. In general, my experience has been that changes to a response scale can be done with minimal threats to validity (e.g. looking to change the 4 item Likert type scale to a 6 item Likert type scale). On the other hand, changing question wording itself usually can't be done "without losing validity" ...


5

Murphy & Cleveland (1995) mention, that a good way to reduce rater errors in general is to inform raters of the existence and nature of these errors and then to simply urge to avoid them. While this reduces rater errors, it also decreases the accuracy of ratings, though. These findings come from the literature on performance assessment, where halo is ...


4

Here are a couple of suggestions: Inquisit records the timing of each response and can be run online. If you care about timing a lot and you need to run the study online, then this is quite a good option. Qualtrics can record various features of survey delivery time. In particular, you can record time spent on each survey page. If you have one question per ...


4

The best review of experience sampling tools I've found is here. Specifically, to answer you question, check out "MyExperience". To quote the website: MyExperience is a BSD-licensed open source mobile data collection tool developed for Windows Mobile devices (including PDAs and mobile phones) using .NET CF 2 and Microsoft SQL Compact Edition. ...


4

You seem to be concerned with reliability, and more specifically internal reliability. Internal reliability is the degree to which different questions are measuring the same construct. This concept is used often in psychology and is usually measured using Cronbach's alpha. However, it is typically used to measure the reliability of a test, and not the ...


3

Preventing random responding: An important first step is to think about ways to prevent random responding from occurring in the first place. A few ideas include: administer the survey face to face; have an experimental invigilator present; communicate the importance of the research to participants and the importance of participants taking the research ...


3

Self-report methodology was one of my qualifying exam topics as a doctoral student of social and personality psychology, so I've got a ton of references to offer, but I confess I haven't read most of them very thoroughly (if at all), and I've forgotten where exactly I've read some of this. It's really a very broad topic as well, so I won't list most I know ...


2

This is not directly an answer to your question but, in line with my comments to another answer, my main advice would be “don't worry about it”. Jeromy Anglim's tips are all good but I am still unconvinced that this is an important issue for most people. Since you are new to research, there are probably dozens of other things you should worry about. ...


2

This is just a few thoughts. I agree with @adb that an obvious way to use facebook for conducting surveys is just to provide a link to an external website that runs the survey. There is an interesting article by Tan and colleagues where they talk about using paid advertisements to get targeted participants on Facebook. I thought the following statistics ...


2

This is probably a bit old but there is a free iOS app called the PIEL Survey. It is in the App store. There is a web site which explains how to set up Experience Sampling surveys.


1

There are lots of commonly observed biases in surveys, and a whole field of survey methodology and statistics which investigates them. To get an idea of this, take a look at the table of contents of these journals: http://poq.oxfordjournals.org/content/current http://jssam.oxfordjournals.org/content/current Many people in this field are dealing with large ...


1

I made up a SVG version connected to a hidden html form. It's not perfect, but it may be useful anyway. You can find it here. Note that a sufficient screen resolution is required. Please send me feedbacks if you use it so that I can improve it.


1

Both Carol Dweck and Angela Duckworth have fairly brief tests meant to measure student success and by some extension grades. Dweck's work focuses on how having a "fixed mindset" limits a students ability to grow in a subject and is used to explain why students are so apt to give up when met with challenge. Her quiz would have to be formatted for pen and ...


1

A few options. You could start with an open ended question and then move to closed ended questions for leading brands and perhaps any other options listed in the previously open ended process. You could use an input box that takes text and shows partial matches which people then select (e.g., a bit like in Facebook when selecting a friend)


1

it seems that you can use randomization of options to decrease bias. That is, present every respondent with multiple choices but in completely random order. Also, bias might be directly tested on two groups, say A&B. Only difference will be the order of available answers. This manual seems to be very helpful: Handbook of Survey Research, Ch.9 Question ...


1

The best way to set up a Mechanical Turk longitudinal study is with TurkPrime.com. You can follow-up with users and invite them to take a new survey. As long as they are properly incentivized they will come. I recommend that you inform them at the start that you are running a 10 part study and that those who perform well on each of the 4 minute studies will ...


1

Question: if you have used such softwares to make experiments, what did you think about the process? We use Paradigm (www.paradigmexperiments.com) in our lab and like it quite a bit. You can build most experiments just with the drag and drop builder interface. It also supports Python scripting just to give you more flexibility (and piece of mind in ...


1

Here are links to several lists of common Likert-type response anchors (the first link provides a nice variety): http://www.clemson.edu/centers-institutes/tourism/documents/sample-scales.pdf http://www.rpgroup.org/sites/default/files/Surveys%20Interactive%20Activity%20-%20Examples%20of%20Likert%20scales.pdf And these links provide suggestions regarding ...


1

You could try third party suppliers. There is currently a company offering free facebook surveys. Basically, create an account, create a survey, enter in a bit of information and presto, you have a survey. Try www.hiprewards.com. Building your own survey app for facebook is not a trivial undertaking.


1

This would be easy to implement in Inquisit. Inquisit has a web interface and is particularly designed for branching and randomisation of trials. If the coding sounds too complex, you can always contact Inquisit and pay them to program the task. From my experience, the cost is not that great for simple scripts. You can see an example of the Iowa Gambling ...



Only top voted, non community-wiki answers of a minimum length are eligible