Testing in the Style of the Georgia Mafia

by Louie Crew

First appeared in Politics & Education 1.4 (1978).

© 1978 by Politics & Education; © 2004 by Louie Crew

The last few months have done much to shake my faith in reason as a vital force in higher education. Perhaps it is just as well that I have a clearer picture of what principles really count with our reigning powers.

I teach in the English Department of one of the three predominately black institutions in the University System of Georgia. Last September I volunteered to teach two sections of composition in our Special Studies Department, a program of non-credit remediation in mathematics and English. The program, now in its fifth year in the System, is administered by Dr. Charles Nash in the Chancellor's office and by local chairpeople of Special Studies, who are independent of the chairpeople of English and mathematics on their respective thirty-one undergraduate campuses.

Until last fall the System used the College Guidance Placement Test (CGP) prepared by the folks at Educational Testing Service (ETS) in Princeton. Effective since December 1977, all units have used the System's own new Basic Skills Examination (BSE), both as a placement test and as an exit examination for those found to need the remedial program. In neither the CGP nor the BSE is there a required writing sample. Both tests are alleged to measure writing ability by the mechanism of multiple choice questions, sixty of which are the fare on the English portion of the new BSE. All items have been chosen by the System's own Special Studies Testing Committee, composed of a selection of the chairpersons of various Special Studies Departments in the System. As there are no announced criteria, many of these test-designers have had no more than the amateur experience of had designing achievement tests for single classes and have/no professional training at all In the skill of designing competency tests for large populations. This committee hag at its disposal the skill of the head of the Regents' entire testing operation, Dr. Robert Rentz; but Dr. Rentz is a mathematician and takes no responsibility for the substance of the items tested, only for their quantification and mathematical analysis. Even if the English examination were written in Sanskrit, so long as students filled in the answers, the mathematician could discover a quantitative way to discriminate between those who guessed excellently well, medium well, or poorly.

Ostensibly, an advantage of having a local group to prepare the test is the experience of having persons who are close to the students and knowledgeable about their writing; in reality, most of the chairpersons are primarily administrators and often have little current and regular experience of the written work of the students whose basic skills they purport to test. Typically, the chairpersons in charge of the English examination have had advanced training in either literature (literature is no part of the BSE) or professional education (a discipline which could mean almost anything.

An unheralded advantage of having a local group to prepare the test is the System's access to this cheap labor force of people already in their employ, whose only additional compensation is in per diem and travel allowances for the meetings of the committee. Furthermore, having its own copyrighted materials (at a mere $10 a test, saves the System thousands of dollars vis-a-vis the folks in Princeton. Currently the System has 88,292 regular students enrolled in the thirty-one undergraduate programs and an additional 8,041 students (8.3 percent of the total) enrolled in Special Studies Departments. Since all of these students must take a placement test, the Regents' testing program is big business. Additionally, the Regents require another test in English competency, called the "Rising Junior [or Regents'] Examination," which must be passed by all students after they have met their sophomore requirements and before they are allowed to graduate.1

At the limited number of meetings at which the Special Studies Testing Committee actually developed Georgia's new BSE, the Committee never clearly defined what "basic skills" are nor did the Committee set priorities' or specify any degrees of "basic." Is the spelling of to-too-two more basic than the spelling of ratiocination? Is the distinction between compliment and complement as basic as the recognition of a sentence fragment? What skills are not "basic" and therefore reserved for the credit courses in English? Answers to some of these questions and many more would seem absolutely essential before any testers could design a meaningful test of "basic skills." Instead, as Professor Jay Simpson, a member of the Committee, explained to me in writing:

We were...faced with the prospect of producing a substitute [for an earlier scrapped effort] in record time, and in spite of my written and detailed objections, the rest of the committee agreed to adopt the questions left over from the discontinued section of the Regents Test. Clearly since the questions were originally intended for students who have finished English 101 and 102 and seven other courses, they were wholly inappropriate for Special Studies students, be they black or white. But availability won over sensibility, and the result is as you see.
The savings effected by the locally made examination hardly justify this kind of shoddiness. At least seven of the sixty items have muddled directions. At least nine of the items require students to make recondite distinctions in diction (e. g., fewer vs. lesser so often vs. ever so often; between vs. among), often distinctions that have had no consensus among even the professionals for at least the past twenty years, as indicated by standard reference works. In choosing spelling items, the testers largely preferred the jargon of English teachers (such as villainy and allusions to the more basic and common problem words, as in the doubling or singling of final letters before suffixation and the ie-ei distinctions, the ible-able distinctions, and the like. The BSE almost completely ignores the very wide-spread and basic problems of making subjects to agree with their verbs, of using the proper tense markers and other standard verb forms, etc.

We teachers who had spent our time drilling our students on these basic matters of their real need perhaps did our students a grave disservice if our effectiveness i9 to be measured in terms of our students' abilities to exit from our courses by the only route allowed, viz., a passing mark on the BSE. We might have been of far more service simply by giving all class time to recondite questions of diction culled from every grammar handbook we could find, but preferably those published before about 1950, by which time most of our testers had fixed their linguistic prejudice.

The testers have avoided any united outcry from the faculties by setting a System-wide passing score at only 19 correct (one-third) out of the 57 (three additional items were "experimental"). With a sure knowledge of any ten items (hardly an impressive competence), many could be expected to guess effectively at least nine (only 19 percent) of the remaining 47. At my campus, the student with the highest score of all wrote an almost illiterate final examination essay, and one of the most articulate of all students whom I have taught over twenty ears, flunked the test. I estimate that a majority of our passing students did so by guesswork, and now they are filling our credit courses without the very basic skills that supposedly the BSE is to assure.

THE PROTEST

It was quite by accident that I discovered that we who taught the Special Studies writing courses could preview one of the English forms of the BSE. As an adjunctive visitor from the English Department, I usually was not informed of the Special Studies Department meetings; and often those meetings were carelessly scheduled in conflict with English Department meetings, although four of us (40 percent of our English Department) were adjuncts in Special Studies. Mr. Robert Roquemore, the director of our Special Studies Department, is a mathematician and repeatedly complained that we composition teachers could not agree on anything anyway, though we had a different assessment of our own discussions. A very alert regular in the Special Studies Department advised me that she had seen the BSE, was alarmed, and wanted my reactions to the test. With no fanfare, I picked up a copy from a counselor who had been charged to make them available to the faculty with a dutiful vocal warning that we should exercise care in keeping them from the students.

When I discovered the kinds of misdirection that I have noted, I called Dr. Nash at the Board of Regents, explained that I had a copy of the test, and asked about the procedures for filing a complaint. He said that the BSE as a new examination would naturally have a few flaws here and there and that my input would be welcome and should be directed to the chairperson of the Special Studies Testing Committee. Knowing that my own written objections were comprehensive and fundamental, not just editorial , I fired off copies o my critique together with confidential copies of the BSE to selected linguists around the country, asking them also to send critiques to the chairperson of the Special Studies Testing Committee, even as I encouraged several of my local colleagues to join the protest. The examination was scheduled to be given for the first time in December; and our critiques started arriving in late October, calling for the immediate withdrawal of the examination on the basis of its documented inaccuracies and misdirections.

At this time Dr. Nash called Mr. Roquemore and asked why other blacks always had to be the only ones to give him trouble: "Can't we take tests just like anybody else?" Thereupon Mr. Roquemore called me in to inquire how I had dared to "jeopardize the security of the BSE" by sending copies around the country. "Shit, man," he yelled, "just as I'm at last about to get my fellow niggers under control, you bring up this problem!" When at his request I explained in writing the limited restrictions which had been placed on my access to the copy of the BSE, noting that my out-of-state linguist correspondents were not likely to send copies to Georgia remedial students, he wrote back: .... You claim a terminal degree. Surely, Dr. Crew, in two decades of teaching, you have been involved in many testing situations as test taker and test giver. Yet, in your letter you claim an understanding of test security that would make a grade school child's knowledge of the subject superior by comparison.

If your statement is a true assessment of your understanding of test security, and I have no reason to doubt your word, I beg your forgiveness for raising complicated questions about impropriety which you have no doubt found incomprehensible....

We have worked extremely hard at developing a first-class testing operation. Your actions have possibly been injurious to us.

That is the way that bullies talk in the back woods. Actually I own up to having very little knowledge about test security matters and am especially confused about the director's propriety in making advanced copies of the test available to anyone. I strongly suspect that he was not supposed to have done so, and my action led to his having been caught. I do know of at least one teacher who taught items directly from the test for the rest of the term, with the disturbing consequence that the students given such prior exposure still did not do any better than the students of the rest of us: apparently the virulence of the test's poison won't even vaccinate!

The head of the Testing Committee responded promptly: I have shared your comments and criticism with the members of my committee.... Since we have received several suggestions regarding the test from around the state, we plan to have a meeting of the sub-committee on English early next quarter in order to review the pool of items and make any necessary changes.

[signed] George W. Brannon

Interestingly Professor Brannon did not add that some of his "several suggestions" had come from outside the state. James Sledd, noted grammarian at the University of Texas, had written a three-page item analysis of the BSE, prefaced: "I don't want to be a busybody. This test, however, is so very bad, and the social injustice which it promises to do so very great, that a native Georgian can't keep polite silence." Dr. Sledd claimed that the "direction writer are prime candidates for the bonehead section" and complained of numerous worthless questions, observing finally:
By my count, then, one-fourth of the questions on this "basic skills examination" display no basic skills. But the strongest objections must still be made. First, such examinations lead innocent students to believe that good writing is a business of carefully avoiding imaginary errors. Second, such examinations guarantee that the vast majority of your minority students will be condemned to bonehead English no matter how intelligent they may be, no matter how powerfully some of them may write. I am compelled to pose a dilemma: either this examination is intended as a form of racial discrimination, or its makers are go thoughtless or incompetent that they have invented a device for discrimination in all innocence.

I urge you most earnestly, Professor Brannon to do all in your power to have this iniquitous examination immediately withdrawn.

Dr. Carolyn Bell at Randolph-Macon Women's College came to a similar conclusion: "Although I consider myself a conservative stylist and composition teacher, I do not believe this test will be useful for placement, and I urge you to replace it."

Dr. Dwayne Stragheim at Hastings College and a past president of the Nebraska Council of Teachers of English, stressed:

As an experienced English teacher and as a local school board member and chairman of a school district curriculum committee which is currently grappling with a similar problem..., I think your efforts have been seriously misguided, and I urge you to scrap your Basic Skills Examination, to solicit the advice and assistance of some competent teachers of writing, and to start over again.

But of course they did not scrap the BSE. They decided instead to consider only the particulars and to ignore the glaring generality in the support of which we had supplied the critiques. If the public had half of our amount of evidence to suggest that a particular soda pop would possibly cause the back ache, the soda pop would be banned until further tests confirmed or disproved the preliminary findings. apparently the University System of Georgia is not equally concerned about the people it holds in public trust.

Failing to get a hearing by behaving collegially, I went to the press. A reporter for the Macon Telegraph gave us a front-page headline: "EXPERTS PROTEST TEST AT FORT VALLEY STATE." (Actually, the "experts" are from all over, and the test is used throughout the state, but folks at the predominately white schools have little commitment to remedial education anyway and could care less.) While the article took basically a strong stand for us, the only clear result has been the increased rudeness of secretaries as they fend me off when I try to get routine information from the test's administrators. As of this writing the testers are refusing to supply copies to a national review board set up by SLATE, a committee sponsored by NCTE and charged to deal with the abuses of competency testing; long ago I had to return my own copy of the BSE and I have been denied further access to it. All of my assignments will be in the English Department hereafter, by a rather mutual, unwritten agreement. The Georgia chapter of ACLU briefly considered a request for a suit against the Regents over the BSE, but concluded:

[Our attorney] says a challenge of the testing procedures would have no chance in federal court. Basically the problem is that the courts in legal cop-outs similar to the end of Reconstruction period, are saying that in discrimination cases you not only have to prove discriminatory effect, but intent to discriminate as well. That is virtually impossible to do.

His only suggestion was to continue to raise the testing question with HEW, trying to get them to make that a part of their approval/disapproval.

Professor Jay Simpson explained to me that at the meeting of the Special Studies Testing Committee at which our critiques were to have been considered there was no attempt to consider them or eve to share with the Committee members the fact of their existence, except for Professor Simpson's own mention of them, whereupon Dr. Nash referred to all other who had written as "Louie Crew's cronies." The one concession, if such indeed it was, was that the Committee discontinued the form of the BSE which we had actually critiqued, keeping another two forms, which are made on the same model and which have many of the same items in a different order. Professor Simpson himself resigned from the Committee in protest. None of these matters has been officially reported to those of us who made the critique.

No real changes have been made. It is as if our protest never really happened. We have been denied even the semblance of due process. The Testing Committee has no clear channels of appeal; and there is no one who takes responsibility for the authorship of the materials. The authorities who administer the BSE never explain themselves. Worse, few seem to care, except some of the student victims; and they are voiceless.

Mr. Roquemore, my local director, has recently tried to be nice to me; and I have suspected all along that his bullying was merely a manifestation of his own fright. He and Dr. Nash alike are but expendable heads of the huge octopus, the State's educational bureaucracy. Simplistic notions of accountability may work well enough in the commercial marketplace, but they play havoc when the Regents, who are mainly businessmen, import them into the learning place. In many ways the entire concept of "basics" has just the right amount of ambiguity to make it ideal for the enslavement of educational bureaucrats.

The bureaucratic octopus is not confined to Georgia. In March of 1977 the Board of Higher Education of New Jersey appointed a "Basic Skills Council"; and with record speed, by the fall, in conjunction with ETS, the group had copyrighted "The New Jersey Placement Tests," with one test on sentence structure and another on logical relationships as the staple for verbal placement. Both of these tests are as recondite and misdirected as Georgia's BSE, in spite of their greater appearance of sophistication.2 Many other states have mandated or are considering mandating such tests.

I have intentionally skirted the issues of whether we should require standard edited English and of whether, if we do, we ought to allow a delay by waiting until a student arrives at college before confronting the student with such a requirement.3 I have likewise largely skirted here James Sledd's strong claim--with which even the testers would likely agree--that minority students will routinely fare less well on these tests. My point has been momentarily to give the testers their advantage and to consider their product on their own logic. The BSE fails flagrantly when thus considered, as in my view do the "New Jersey Placement Tests." James Sledd's comment about the latter fits them both: "Tests like this overwhelmingly suggest relatively dull middle-class whites sitting in a room dreaming up improbable sentences to catch students out. " In view of who pays our checks, that is probably exactly what we are hired to be and do.

NOTES

1See my "Lawd, Hare Mercy, Ms. Scarlet!" ( Black Times, vol. 4, no. 8 [August 1974], 7) and my "The New Alchemy" (College English, vol. 38 no. 7 [March 1977], 707-711) for a fuller critique of this Rising-Junior Examination. When I urged the chancellor to take advantage of College English "Comment an Rebuttal" section to reply to the charges which I had raised, he disclaimed responsibility for the test and referred me to the English Committee, made up of the chairpeople of all departments of English in the System. When I wrote their Committee chairperson, he ever gave me even the courtesy of a reply.

2 For materials on the New Jersey test, see The New York Times for November 27, 1977, "Basic-Skills Tests are Called 'Elitist"'; Chancellor Hollander's defense, "The Open College Door," The New York Times, December 18, 1977; and additional letters in The New York Times for December 25, 1977 and January 15, 1978.

3 The major document is "The Students' Right to Their Own Language," adopted by the Conference on College Composition and Communication and printed with supporting materials in CCC, vol. 26 (September 1974). I have articles of my own in CCC, vol. 25 (February 1974), College English, vol. 34, no. 4 (January 1973) and vol. 35, no. 4 (January 1974); Exercise Exchange, vol. 18, no. 2 (Spring 1974); and Phylon, vol. 36, no. 2 (1975). 


------------------------------------

Please sign my guestbook and view it.


My site has been accessed  times since February 14, 1996.

Statistics courtesy of WebCounter.