[Federal Register: June 25, 2009 (Volume 74, Number 121)][Proposed Rules]
[Page 30250-30256]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr25jn09-20]
=======================================================================
-----------------------------------------------------------------------
DEPARTMENT OF LABOR
Occupational Safety and Health Administration
29 CFR Part 1910
[Docket No. OSHA-2007-0006]
RIN 1218-AC29
Abbreviated Bitrex® Qualitative Fit-Testing Protocol
AGENCY: Occupational Safety and Health Administration (OSHA); Labor.
ACTION: Proposed rule; withdrawal.
-----------------------------------------------------------------------
SUMMARY: After thoroughly reviewing the comments and other information
available in the record for the proposed rulemaking, OSHA decided that
the abbreviated Bitrex® qualitative fit test is not sufficiently
accurate to include among the qualitative fits tests listed in Part II
of Appendix A of its Respiratory Protection Standard. Therefore, OSHA
is withdrawing the proposed rule without prejudice, and is inviting
resubmission of the proposed fit test after conducting further research
to improve the accuracy of the protocol.
DATES: Effective June 25, 2009, the proposed rule published December
26, 2007 (72 FR 72971) is withdrawn.
FOR FURTHER INFORMATION CONTACT: General information and press
inquiries: Contact Ms. Jennifer Ashley, Office of Communications, Room
N-3647, OSHA, U.S. Department of Labor, 200 Constitution Avenue, NW.,
Washington, DC 20210; telephone (202) 693-1999.
Technical inquiries: Contact Mr. John E. Steelnack, Directorate of
Standards and Guidance, Room N-3718, OSHA, U.S. Department of Labor,
200 Constitution Avenue, NW., Washington, DC 20210; telephone: (202)
693-2289; facsimile: (202) 693-1678. Electronic copies of this Federal
Register notice, as well as news releases and other relevant documents,
are available at OSHA's Web page at http://www.osha.gov.
SUPPLEMENTARY INFORMATION:
I. Background
Part I to Appendix A of OSHA's Respiratory Protection Standard at
29 CFR 1910.134 currently includes four qualitative fit-testing
protocols using the following challenge agents: Isoamyl acetate;
saccharin-solution aerosol; Bitrex® (denatonium benzoate)
aerosol in solution; and irritant smoke (stannic chloride). Part II to
Appendix A specifies the procedure by which OSHA determines whether to
propose adding a new fit-testing protocol to the Respiratory Protection
Standard. The criteria used in making this determination include: (1) A
test report prepared by an independent government research laboratory
(e.g., Lawrence Livermore National Laboratory, Los Alamos National
Laboratory, the National Institute for Standards and Technology)
stating that the laboratory tested the protocol and found it to be
accurate and reliable; or (2) an article published in a peer-reviewed
industrial-hygiene journal describing the protocol and explaining how
the test data support the protocol's accuracy and reliability. If a
fit-testing protocol meets one of these criteria, OSHA must initiate
notice-and-comment rulemaking on the proposed fit-testing protocol
under Section 6(b)(7) of the Occupational Safety and Health Act of 1970
(29 U.S.C. 655).
II. Summary and Explanation of the Withdrawal Notice
A. Introduction
In the letter submitting the abbreviated Bitrex® qualitative
fit-testing (ABQLFT) protocol for review under the provisions of
Appendix A of OSHA's Respiratory Protection Standard (Ex. OSHA-2007-
0006-0002), Dr. Michael L. Runge of the 3M Company included a copy of a
peer-reviewed article from an industrial-hygiene journal describing the
accuracy and reliability of the ABQLFT protocol (Ex. OSHA-2007-0006-
0003). This article also described in detail the equipment and
procedures required to administer the ABQLFT protocol. According to
this description, the protocol is a variation of the existing
Bitrex® qualitative fit-testing protocol developed by the 3M
Company in the early 1990s, which OSHA approved for inclusion in the
final Respiratory Protection Standard. The ABQLFT protocol uses the
same fit-testing requirements and instrumentation specified for the
existing Bitrex® qualitative fit-testing protocol in paragraphs
(a) and (b) of Part I.B.4 of Appendix A of the Respiratory Protection
Standard, with the following two exceptions:
Exercise times are reduced from 60 seconds to 15 seconds;
and
The ABQLFT protocol is used only with test subjects who
can taste the Bitrex® screening solution within the first 10
squeezes of the nebulizer bulb (referred to as "Level 1
sensitivity").
The peer-reviewed article submitted by the 3M Company describing
the study conducted on the ABQLFT, entitled "Development of an
Abbreviated Qualitative Fit Test Using Bitter Aerosol," appeared in
the Fall/Winter 2003 issue of the Journal of the International Society
for Respiratory Protection (hereafter, "the ABQLFT study" or "the
study"; Ex. OSHA-2007-0006-0003). The authors of the study were T.J.
Nelson of NIHS, Inc., and L.L. Janssen, M.D. Luinenburg, and H.E.
Mullins of the 3M Company; the 3M Company supported the study. The
study described by the article determined whether performing a fit test
involving seven exercises lasting 15 seconds each while exposed to
Bitrex® solution aerosol yielded fit-testing results similar to
results obtained with a generated-aerosol (i.e., corn oil) quantitative
fit test (GAQNFT) using one-minute exercises (i.e., the GAQNFT was the
criterion measure or "gold standard").
The study involved 43 experienced respirator users, 20 females and
23 males. The test subjects followed the existing Bitrex®
qualitative fit-testing protocol in Appendix A of OSHA's Respiratory
Protection Standard except that they performed each of the fit-testing
exercises for 15 seconds (instead of 60 seconds) while wearing a NIOSH-
certified elastomeric half-mask respirator equipped with P100 filters.
The authors selected the best fitting respirator for each test subject
from among four models, each available in three sizes; some test
subjects used more than one model during fit testing. In addition, the
authors induced poor respirator fits by assigning a respirator to test
subjects that was one or two sizes too small or too large as determined
by the Los Alamos National Laboratory panel-grid size and observation
of the test subjects' facial characteristics. Test subjects could
adjust the respirator facepiece for comfort, but they did not perform
user seal checks.
In conducting the study, the authors used the recommendations for
evaluating new fit-test methods specified by Annex A2 of ANSI Z88.10-
2001, including sequencing the ABQLFT and GAQNFT in random order
without disturbing facepiece fit. The authors used fit-test sample
adaptors or respirators with fixed probes to collect samples inside the
respirator. The sample point inside the respirator was located between
the nose and the mouth. For both fit tests, the authors had the test
subjects perform seven of the eight exercises listed in Part I.A.14 of
Appendix A of OSHA's Respiratory Protection Standard, which included:
Normal breathing, deep breathing, turning the head side to side, moving
the head up and down, reading a passage, bending over, and normal
breathing.\1\ For the GAQNFT, the authors performed particle counts at
one-second intervals inside a test chamber for 15-30 seconds before and
after fit testing, and inside the respirator for the 60-second duration
of each exercise.
---------------------------------------------------------------------------
\1\ The test subjects did not perform the grimace exercise.
---------------------------------------------------------------------------
The 43 test subjects used in the study had Level 1 sensitivity to
Bitrex® because they were able to taste the Bitrex®
aerosol within 10 squeezes of the nebulizer bulb. Subjects having Level
2 or 3 sensitivity to Bitrex® were excluded from further
participation in the study because the nebulizer could not be
replenished for additional taste testing within the 15 seconds allotted
to perform each fit-testing exercise. After the test subjects passed a
Bitrex® sensitivity-screening test, the authors administered the
ABQLFT using the procedures and techniques specified for the existing
Bitrex® qualitative fit-testing protocol in Part I.B.14 of
Appendix A of OSHA's Respiratory Protection Standard, and determined
the fit factor using the particle count for the 15-second duration of
each exercise.
The authors required a fit factor of 100 to pass a fit test, which
served as the basis for determining the following statistics for the
ABQLFT: Test sensitivity; predictive value of a pass; test specificity;
and predictive value of a fail. In calculating these statistics, the
authors adopted the variables defined by ANSI Z88.10-2001, in which: A
= false positives (passed the fit test with a fit factor < 100); B =
true positives (passed the fit test with a fit factor >= 100); C = true
negatives (failed the fit test with a fit factor < 100); and D = false
negatives (failed the fit test with a fit factor >= 100). Using these
variables, ANSI Z88.10-2001 specifies the formula and recommended value
(RV) for each statistic as follows: Test sensitivity = C/(A + C), RV >=
0.95; predictive value of a pass = B/(A + B), RV >= 0.95; test
specificity = B/(B + D), RV > 0.50; and predictive value of a fail = C/
(C + D), RV > 0.50.
Using the GAQNFT as the criterion measure, the variables for the
ABQLFT had the following values: A = 4; B = 95; C = 48; and D = 20. The
statistics calculated for the ABQLFT from these values were: Test
sensitivity = 0.92; predictive value of a pass = 0.96; test specificity
= 0.83; and predictive value of a fail = 0.71. Therefore, every
statistic for the ABQLFT, except test sensitivity, attained a value in
excess of the ANSI Z88.10-2001 recommended value.
The test-sensitivity value of 0.92 for the ABQLFT fell below the
ANSI recommended value of 0.95. The authors state that this slight
difference represents a single false positive value for the ABQLFT
(i.e., failed the GAQNFT but passed the ABQLFT). However, an additional
peer-reviewed article submitted by Dr. Runge of the 3M Company suggests
an alternative approach to examining these test-sensitivity values (see
Ex. OSHA-2007-0006-0004). This article, entitled "Recommendations for
the Acceptance Criteria for New Fit Test Methods" and published in the
Spring/Summer 2004 issue of the Journal of the International Society
for Respiratory Protection, describes an analytical study conducted by
T. J. Nelson of NIHS, Inc. and H. Mullins of the 3M Company, and
supported by the 3M Company. In this study, the authors performed a
binary logistic-regression analysis on pass-fail fit-testing data from
published studies involving two quantitative, and two qualitative, fit
tests. The authors justify using the binary logistic-regression
analysis for this purpose as follows:
When a simple sensitivity test is used to describe a new test,
the result can be affected by the distribution of the data. In
several cases using the theoretical distributions described in this
paper, the outcome of a sensitivity test for the Bitrex and Ambient
Particle Counter fit tests could have failed to meet the ANSI Z88.10
sensitivity requirement. The method used to determine acceptability
should be independent of specific data collected. (See Ex. OSHA-
2007-0006-0004, p. 8.)
The results of the binary logistic-regression analysis performed on
the ABQLFT data showed that the ABQLFT had a 0.20 probability of
passing a respirator user with a fit factor of 50 and a 0.33
probability of passing a respirator user with a fit factor of 100.
Figure 3 of the article compares the binary logistic-regression
analysis results of test-sensitivity values obtained for a popular
quantitative fit test and the existing 60-second Bitrex®
qualitative fit test. The authors conclude that the analysis
demonstrates that the distribution of fit-testing data affected the
test-sensitivity values derived using the ANSI Z88.10-2001 test-
sensitivity calculations. Based on this analysis, the authors assert
that "a sensitivity calculation may not be the best indicator of fit
test method performance. The binary logistic regression analysis shows
that the result of the 15 second exercise time test is very similar to
the ambient aerosol and 60 second bitter aerosol tests" (Ex. OSHA-
2007-0006-0003, p. 108). In summarizing the results, the authors state
that "[t]he 15 second bitter aerosol protocol sufficiently screens for
adequate respirator fit in subjects with Level 1 Bitrex taste
sensitivity."
After carefully reviewing the peer-reviewed articles submitted in
support of the ABQLFT, OSHA determined that the protocol met the second
criterion specified in Appendix A of the Respiratory Protection
Standard, and then developed a proposal to add a new fit-testing
protocol to the standard. OSHA published the proposal in the Federal
Register on December 26, 2007 (see 72 FR 72971).
B. Issues Raised for Public Comment
In the Federal Register notice announcing the proposal, OSHA
invited comments and data from the public regarding the accuracy and
reliability of the proposed ABQLFT protocol, its effectiveness in
detecting respirator leakage, and its usefulness in selecting
respirators that will protect employees from airborne contaminants in
the workplace. Specifically, the Agency invited public comment on the
following issues:
Were the studies described in the submitted articles well
controlled, and conducted according to accepted experimental design
practices and principles?
Were the results of the studies described in the submitted
articles properly, fully, and fairly presented and interpreted?
Will the proposed ABQLFT protocol generate reproducible
fit-testing results, and what additional experiments or analyses of
existing data are necessary to answer this question?
Will the proposed ABQLFT protocol reliably identify
respirators with unacceptable fit as effectively as the qualitative
fit-testing protocols, including the existing Bitrex®
qualitative fit-testing protocol, already listed in Part I.B of
Appendix A of the Respiratory Protection Standard?
What is the significance of the test-sensitivity value of
0.92 obtained for the ABQLFT relative to the test-sensitivity value of
0.95 recommended by ANSI Z88.10-2001, and does the authors' assertion
that "a sensitivity calculation may not be the best indicator of fit
test method performance" adequately account for the lower test-
sensitivity value?
What is the significance of limiting the ABQLT to
respirator users who demonstrate Level 1 sensitivity to Bitrex®?
C. Summary of the Public Comments Received
Twenty-two commenters submitted responses to the proposal. The
following paragraphs in this section address the responses made to each
of the six issues described previously, as well as additional issues
addressed by the commenters themselves.
1. Were the studies described in the submitted articles well
controlled, and conducted according to accepted experimental design
practices and principles? In addressing this issue, NIOSH stated:
The primary journal article cited, Development of an Abbreviated
Qualitative Fit Test Using Bitter Aerosol by Nelson et al. [2003],
does not provide sufficient detail about the study design and
protocol to enable a complete assessment of how well it was
controlled and conducted. The description in the article does
indicate that design and principles met acceptable practices. (See
Ex. OSHA-2007-0006-0026.)
Jeff Weed asserted that the study did not exclude from the statistical
analysis the fit factors used to determine the reference-method fit
factors within one standard deviation of the required fit factor, a
determination required under ANSI Z88.10-2001 (Ex. OSHA-2007-0006-
0020.1).
Generally, the NIOSH comment appears to support the design
practices and principles used in the study, and did not elaborate on
what additional detail would "enable a complete assessment of how well
[the study] was controlled and conducted." Jeff Weed's comment appears
to be mistaken because page 104 of the article describing the study
(see Ex. OSHA-2007-0006-0003) states that the "[f]ive fit factors
within one standard deviation of the required fit factor of 100 (86 to
114) were excluded from the data analysis as recommended by Z88.10."
Therefore, OSHA concludes that the study was well controlled, and
conducted according to accepted experimental design practices and
principles.
2. Were the results of the studies described in the submitted
articles properly, fully, and fairly presented and interpreted? NIOSH
made the following comments regarding this issue:
NIOSH is concerned that the interpretation of the study results
does not appropriately represent the performance of the fit testing
protocol. The authors correctly stated that a shortened bitter
aerosol fit test method relies on two assumptions: (1) Fit does not
significantly change during an exercise and (2) people being tested
will respond to the bitter taste of Bitrex® in the shorter
time period. The results of the study support the second assumption,
i.e., the test subjects classified with Level 1 sensitivity
responded to the bitter taste of Bitrex® in the shorter time
period. However, the study results do not provide convincing
evidence to support the first assumption. * * *
The consistency of the respirator's fit throughout each of seven
exercises is important in the assessment of the performance of the
ABQLFT fit test protocol. The fit factor assigned for each ABQLFT
exercise in the study is based on a 15-second increment, in contrast
to a 60-second increment for each of the same exercises performed in
quantitative fit test (GAQNFT) protocol. Change in fit during an
exercise suggests that the fit at the start of the next 60-second
exercise in the GAQNFT is more likely to differ from the fit at the
start of the corresponding 15-second exercise period of the ABQLFT.
There is no indication that the authors considered the significance
of the noted changes in fit on the accuracy of the assigned fit
factors. (See Ex. OSHA-2007-0006-0026.)
Pages 104, 105, and 107 of the article describing the study (see Ex.
OSHA-2007-0006-0003) addressed NIOSH's concerns about the variability
of respirator fit for the 15-second and 60-second exercise periods, at
least for the GAQNFT. Page 104 of the article states that the
correlation between fit factors assessed for the two exercise periods
was highly significant, with r = 0.97, while the text and figure on
page 108 of the article note that variability was low for fit factors
less than 100 and over 6,000. These results demonstrate convincingly
that respirator fit factors, especially for fit factors in the range of
interest (i.e., having values at and below 100), were reasonably
consistent and stable across the 15-second and 60-second exercise
periods.
Jeff Weed commented (see Ex. OSHA-2007-0006-0020.1) that the study
did not report a Kappa value, which ANSI Z88.10-2001 defines as the
"statistic (K) used to calculate some degree of agreement between two
fit tests"; the ANSI standard recommends a minimum Kappa value greater
than 0.70. Based on the equation for the Kappa statistic provided in
Annex A2 of the ANSI standard, Mr. Weed calculated the Kappa value for
the study data as 0.69, which corresponds closely to our calculation of
0.70, rounded from a figure of 0.69565. OSHA concludes, that the Kappa
value calculated from the study data indicates an acceptable degree of
agreement between the two fit tests used in the study, and conforms
satisfactorily with the value recommended by the ANSI standard.
3. Will the proposed ABQLFT protocol generate reproducible fit-
testing results, and what additional experiments or analyses of
existing data are necessary to answer this question? NIOSH questioned
the reproducibility of the fit-testing results, stating:
Based on review of Nelson et al. [2003] and Nelson and Mullins
[2004], NIOSH concludes that the evidence is inadequate to
demonstrate reproducible fit testing results. Further investigation
is required to compare potential changes in fit across the proposed
15-second exercise intervals in the ABQLFT protocol and the standard
60 second exercise intervals in the GAQNFT protocol. At a minimum,
the frequency and consistency of leaks during each exercise, as well
as the magnitude and type of those leaks (e.g. start of exercise,
end of exercise, throughout exercise period) need to be identified
and analyzed. (See Ex. OSHA-2007-0006-0026.)
OSHA addressed NIOSH's concern regarding the variability of respirator
fit for the 15-second and 60-second exercise periods above (see item
C.2 of this section).
Jeff Weed questioned whether employers could reproduce the results
of the ABQLFT study in the workplace, stating:
When qualitative fit test (QLFT) methods such as the ABQLFT are
performed in a laboratory by researchers, the results are reasonably
reproducible. Researchers are keenly aware of the potential mistakes
that cause variability, such as the manner in which the nebulizer bulb
is squeezed (e.g. fully vs. partly, with the palm vs. the fingers, slowly
vs. quickly). The way the nebulizer is used has a significant affect on
the mass of agent that is injected into the fit test hood. Unfortunately,
studies such as the one by Nelson do not take the practicality of the fit
test method into account, when implemented by lay-persons. (See Ex. OSHA-2007-
0006-0020.1.)
The authors of the ABQLFT study mention on page 103 of the article
describing the study (see Ex. OSHA-2007-0006-0003) that "[t]he bitter
aerosol fit test followed the procedure outlined in the OSHA respirator
standard, except that a 15 second exercise period was used." Section
B.4 of Part I in Appendix A of that standard describes in elaborate
detail how to administer properly the Bitrex® solution aerosol
using the nebulizer bulb. OSHA holds that this description of the
procedure is adequate, and that employers are responsible for complying
fully with the procedure as described in OSHA's Respiratory Protection
Standard. In addition, Mr. Weed's comment appears to be speculative in
that he provided no evidence to support it.
Ching-tsen Bien mentioned that "[t]here is only one repeated test
on the same test subject with a standard deviation of 14" (Ex. OSHA-
2007-0006-0017.1). In a response to Mr. Bien, Robert A. Weber of 3M
stated (see Ex. OSHA-2007-0006-0021.1) that Mr. Bien's comment
describes the requirement specified in Annex A2 of ANSI Z88.10-2001.
Mr. Weber quotes this requirement from Annex A2 as follows: "One
standard deviation for the reference method can be approximated by
identifying a subject having a fit factor near the required fit factor
and making measurements on this subject during a single mask donning to
determine system reproducibility." OSHA believes that Mr. Weber's
response appropriately addresses Mr. Bien's concern.
4. Will the proposed ABQLFT protocol reliably identify respirators
with unacceptable fit as effectively as the qualitative fit-testing
protocols, including the existing Bitrex® qualitative fit-
testing protocol, already listed in Part I.B of Appendix A of the
Respiratory Protection Standard? Pete Stafford of the Building and
Construction Trades Department, AFL-CIO, questioned whether the 15-
second exercise periods prescribed by the proposed ABQLFT protocol were
sufficient to challenge the face-to-facepiece seal, stating:
In the abbreviated protocol, normal and deep breathing exercises
would only allow four to five breaths in 15 seconds. Side to side
and up and down exercises might only allow one cycle of each in 15
seconds. The talking exercise would be difficult to accomplish, as
the rainbow passage presents a variety of facial expressions, and
could not be completed in the 15 second time frame." (See Ex. OSHA-
2007-0006-0024.)
NIOSH argued that with the aerosol concentration replenished only once
every 30 seconds, the exercise occurring during the first 15 seconds of
this 30-second period would be near the maximum aerosol concentration,
while the exercise occurring during the last 15-second period would be
near the minimum concentration that occurs after filtration removes
much of the aerosol from the hood. NIOSH further noted:
* * * [T]he 60-second exercise duration in the OSHA-accepted
Bitrex® protocol would be conducted through two complete 30-
second concentration-cycles, whereas the 15-second exercises of the
ABQLFT were conducted through only half of one. While the variation
in the aerosol concentration during this procedure has not been
documented, the fact that the replenishing amount is half the
quantity to establish the appropriate test challenge (for a fit
factor of at least 100) suggests that variability could
significantly affect the results. In addition, the variability in
subjects' ability to taste Bitrex® at reduced concentrations,
and the impact on the pass/fail results, needs to be determined and
analyzed. (See Ex. OSHA-2007-0006-0024.)
OSHA finds that the comments submitted by both Pete Stafford and
NIOSH did not adequately consider the effects the alleged deficiencies
should have on the results of the ABQLFT study. Failure to adequately
challenge the facepiece-to-face seal, and low levels of aerosol present
during an exercise, should increase the number of false positives, but
the study data show no such effect. Therefore, absent any supporting
data or analyses, OSHA considers these comments to be speculative.
A number of commenters stated that the proposed protocol would not
reliably assess proper fit for filtering-facepiece respirators because
the authors did not include these respirators in the study design. In
this regard, NIOSH noted that "the submitted study did not include any
filtering facepiece respirators. This type of respirator is commonly
used and likely to be evaluated by the ABQLFT protocol. NIOSH
encourages evaluation of filtering facepiece respirators before
acceptance of the ABQLFT protocol" (Ex. OSHA-2007-0006-0026). Ching-
tsen Bien asserted that "the validation testing should be performed on
a variety of shapes of N-95 filtering facepieces to ensure that this
method would reject inadequate fits for respirators of this type" (Ex.
OSHA-2007-0006-0017.2).
OSHA received additional comments on this issue from Timothy
Roberts, who stated, "Another major concern is that the primary
article [Nelson, 2003] did not include filtering facepiece respirators
as part of the tests. Filtering facepiece respirators are often tested
with the Bitrex qualitative protocol and therefore, the data may not be
representative of the adequacy of the ABQLFT proposal for this class of
respirators" (Ex. OSHA-2007-0006-0022). James S. Johnson recommended
further testing of filtering facepieces using the proposed ABQLFT
protocol, noting, "A similar study (Article 1) needs to be done with
filtering facepiece respirators to demonstrate acceptable performance
is achieved with this type of half mask respirator" (Ex. OSHA-2007-
0006-0028).
Robert Weber of 3M addressed the issue of testing filtering-
facepiece respirators in his comments (see Ex. OSHA-2007-0006-0021.1),
stating, "It is not possible to use N95 filtering facepieces to
validate a fit test with submicrometer particle QNFT," adding that
"[i]t is an evaluation of facepiece[-]to-face seal only; filter
penetration is not included. While filter penetration of submicrometer
particles through N95 filters is small, it is not zero." Mr. Weber
concludes, "The use of N95 [filtering-facepiece respirators] would
therefore skew the data by increasing [false-negative] error, i.e.
rejecting adequate fits."
Contrary to Mr. Weber's comments, OSHA finds that testing N95
filtering-facepiece respirators as recommended by the other commenters
is not validation testing, but instead is testing that would
demonstrate that the proposed ABQLFT protocol performs adequately with
N95 filtering-facepiece respirators, even when filter penetration
increases false-negative error. Therefore, OSHA could not approve using
the proposed ABQLFT protocol for fit testing filtering-facepiece
respirators absent appropriate results demonstrating that the proposed
protocol adequately determines fit for these respirators.
5. What is the significance of the test-sensitivity value of 0.92
obtained for the ABQLFT relative to the test-sensitivity value of 0.95
recommended by ANSI Z88.10-2001, and does the authors' assertion that
"a sensitivity calculation may not be the best indicator of fit test
method performance" adequately account for the lower test-sensitivity
value? In addressing the first part of this issue (i.e., the
significance of the test-sensitivity value of 0.92), Jeff Weed stated,
"[I]t should be noted that of the 5 ANSI criteria, test sensitivity is
the only one that ANSI states `shall' be met. The others carry the
`should' qualifier. In ANSI parlance (paragraph 1.3), the word `shall'
implies a mandatory provision, and `should' is used for advisory
provisions" (Ex. OSHA-2007-0006-0020.1). Similarly, Bill Kajola of the
AFL-CIO stated recommended that OSHA withdraw the proposed rule because
"the most important ANSI criterion for approving a new test method has
not been achieved," and that "[t]he research paper used by 3M in
support of its application for approval (Ex. OSHA-2007-0006-0003)
acknowledges the failure of the 15 second Bitrex fit test protocol to
achieve the ANSI test sensitivity of 0.95 or greater, a consensus
criteria established by the respiratory protection community" (Ex.
OSHA-2007-0006-0019.1). Timothy Roberts, Mark Haskew, and Ching-tsen
Bien stated that failure to achieve the ANSI test-sensitivity criterion
was sufficient justification for OSHA not to adopt the ABQLFT (see Exs.
OSHA-2007-0006-0022, -0023, and 0017.2, respectively). NIOSH believed
that the reduced sensitivity-test value demonstrated that the proposed
ABQLFT protocol was defective, stating, "A sufficient number of
subjects met fit testing requirements using the ABQLFT protocol and
failed using the GAQNFT protocol," and that "[t]he sensitivity test
is a critical criterion to ensure the rejection of inadequately fitting
respirators" (Ex. OSHA-2007-0006-0026). NIOSH concluded that
"[b]ecause the observed value of 0.92 is below the ANSI criterion of
0.95, NIOSH considers the value unacceptable."
In the article describing the ABQLFT study (see Ex. OSHA-2007-0006-
0003), the authors state that "[a]dvisory criteria for evaluating new
fit test methods outlined in Annex A2 to ANSI Standard Z88.10-2001 were
used. * * *" Therefore, the authors adopted the ANSI standard as the
method by which to evaluate the results of the study, including the
test-sensitivity criterion which, as stated above by Mr. Weed, is the
only criterion in the ANSI standard that is mandatory. OSHA believes
adopting the ANSI standard is appropriate because that standard
represents the consensus of the industrial-hygiene community regarding
the criteria to use in assessing fit-testing protocols. The comments
described in the previous paragraph clearly demonstrate that the
industrial-hygiene community generally supports using the ANSI standard
for this purpose.
In comments submitted to the record, Robert Weber of 3M noted that
"there is little significance to the test sensitivity of 0.92 versus a
criterion of 0.95" (Ex. OSHA-2007-0006-0021.1). On page 108 of the
article describing the ABQLFT study (see Ex. OSHA-2007-0006-0003), the
authors observe that "[t]he difference between a sensitivity of 0.92
and a value greater than 0.95 in this comparison is one fit test where
a person with a generated fit factor less than 100 passed the bitter
aerosol fit test." Based on Table 1 in this article, the 0.95
criterion would permit three false-positive test subjects out of 167
subjects tested (i.e., 0.018% of the total subjects tested), while the
obtained value of 0.92 resulted in four false-positive test subjects
(i.e., 0.024% of the subjects tested).
In the NIOSH-Bureau of Labor Statistics survey of respirator use
cited in the proposal (NIOSH-BLS survey; Ex. 6-3, Docket H-049C),
282,000 establishments in the United States required respirator use,
and these establishments fit tested about 3.3 million employees each
year. According to the NIOSH-BLS survey, 18,938 (0.067%) of these
establishments used the existing Bitrex® qualitative fit-testing
protocol.\2\ Assuming that these establishments would substitute the
proposed ABQLFT protocol for the existing Bitrex® qualitative
fit-testing protocol, and that the distribution of employees across
size classes for these establishments is representative of the
establishments as a whole,\3\ then 221,100 employees would receive the
proposed ABQLFT protocol annually (i.e., 0.067% x 3.3 million
employees).
---------------------------------------------------------------------------
\2\ The proposal cited a figure of "approximately 25,000
establishments," but this figure is for the original Controlled
Negative Pressure quantitative fit-testing protocol specified by
OSHA when it first published the Respiratory Protection Standard in
1998, not for the existing Bitrex® qualitative fit-testing
protocol.
\3\ The term "size classes" refers to the number of employees
in the establishments; the NIOSH-BLS survey designates these classes
as follows: 1-10 employees; 11-19 employees; 11-49 employees; 50-249
employees; 250-999 employees; and 1,000 and more employees. A
cursory review of the size-class distribution in the NIOSH-BLS
survey shows that 0.088% of the total number of establishments have
1,000 or more employees, while 0.094% of establishments
administering the existing Bitrex® qualitative fit-testing
protocol have 1,000 or more employees; this comparison indicates
that the distribution of size classes for the latter establishments
is similar to the distribution of size classes for the
establishments as a whole.
---------------------------------------------------------------------------
Under the 0.95 sensitivity-test criterion value for the ANSI
Z88.10-2001 standard, about 3,980 employees with improperly fitting
respirators would pass the proposed ABQLFT protocol each year (i.e., a
0.018% false-positive rate x 221,100 total employees tested), while the
0.92 sensitivity-test value obtained for the proposed protocol would
result in about 5,306 employees passing the test with improperly
fitting respirators (i.e., a 0.024% false-positive rate x 221,100 total
employees tested). OSHA believes that the 3,980 employees with false-
positive values that would result from using the sensitivity-test
criterion from the ANSI standard are too high; therefore, adding 1,326
employees each year to this already excessive figure is unacceptable.
Contrary to the previously cited statement made by Mr. Weber from 3M,
OSHA finds that the significance between test sensitivity values of
0.92 and 0.95, when viewed in practical terms, is highly significant
because an additional 1,326 employees would not have adequate
respiratory protection in the workplace. OSHA believes that the
contribution of ANSI Z88.10-2001 to the process of evaluating proposed
respirator fit-testing protocols is to provide procedures that OSHA can
use in determining the practical effects of errors that result from the
administration of these proposed protocols. Therefore, based on this
analysis involving the sensitivity-test criterion from the ANSI
standard, OSHA concludes that it cannot include the proposed ABQLFT
protocol among the qualitative fit tests currently listed in Part I.B
of Appendix A of its Respiratory Protection Standard.
Regarding the second part of this issue (i.e., that the sensitivity
calculations may not be the best indicator of fit-test performance),
the authors of the study recommended using binary logistic-regression
analysis to determine sensitivity of the proposed protocol instead of
the test-sensitivity criterion specified by ANSI Z88.10-2001. Every
comment submitted to the record opposed this recommendation. For
example, NIOSH stated:
A second cited journal article [Nelson and Mullins 2004]
examined the treatment of data from previously reported studies,
including the 2003 Nelson study, by use of a new method of data
analysis. A more thorough evaluation of the method of data analysis
should be undertaken to ensure the studies used to validate the new
method include an appropriate range of fit factors and respirator
designs.
* * * * *
The argument by the study authors that "the method used to
determine acceptability should be independent of specific data
collected" is not convincing. A sufficient number of subjects met
fit testing requirements using the ABQLFT protocol and failed using
the GAQNFT protocol. These results were determined to be below the
ANSI Z88.10-2001 recommended criteria of 0.95 for the test-sensitivity
value. Recalculating test sensitivity (proportion of failed reference
method fit tests that also failed the new fit-test method) via alternative
statistical techniques, or questioning the validity of the sensitivity
calculation as an appropriate indicator of fit-test method performance to
rationalize a positive conclusion, is a questionable response to the study
outcome. The sensitivity test is a critical criterion to ensure the
rejection of inadequately fitting respirators. Because the observed
value of 0.92 is below the ANSI criterion of 0.95, NIOSH considers
the value unacceptable. If the method of data analysis is changed,
the new method needs to be thoroughly evaluated before challenging
the standard criterion. (See Ex. OSHA-2007-0006-0026.)
Bill Kajola of the AFL-CIO recommended that OSHA not sanction the
binary logistic-regression analysis as an alternate method for
analyzing the study results, stating, "There is no data or
confirmation to suggest that a `binary logistic regression analysis' is
an appropriate and adequate means to evaluate a new fit test method"
(OSHA-2007-0006-0019.1). James S. Johnson believed it was premature to
use binary logistic-regression analysis to analyze the study data,
asserting that "[t]he proposed change is too significant to be based
on one study that has to have additional mathematical analysis and
assumptions proposed to pass the ANSI Z88.10 requirements" (Ex. OSHA-
2007-0006-0028). Daniel K. Shipp of the International Safety Equipment
Association commented that binary logistic-regression analysis "be
validated by an additional source" (Ex. OSHA-2007-0006-0027).
As noted earlier, none of the comments submitted to the record
supported using binary logistic-regression analysis to interpret the
study results. These comments clearly indicate that this analytic
technique is currently inappropriate for use in determining the
sensitivity of fit-testing protocols. OSHA agrees with these comments,
and believes that the technique requires additional validation before
it will be acceptable for this purpose.
6. What is the significance of limiting the ABQLT to respirator
users who demonstrate Level 1 sensitivity to Bitrex®? Few
commenters responded to this issue. NIOSH observed that information
about "the number or percentage of subjects in [the] study who did not
meet Level 1 sensitivity to Bitrex®" was not available in the
article describing the study (see Ex. OSHA-2007-0006-0003), and,
therefore, "NIOSH is unable to estimate the proportion of workers in
the population who demonstrate Level 1 sensitivity to Bitrex®"
(Ex. OSHA-2007-0006-0026). As a result, NIOSH found that "the utility
of the proposed ABQLFT protocol can not be determined at this time."
James S. Johnson commented that determining Level 1 sensitivity is a
restriction that "adds another level of complexity to the test
protocol" (Ex. OSHA-2007-0006-0028). Ching-tsen Bien believed that
using Level 1 sensitivity for screening purposes "does not prevent
some test conductors who ignore this limitation and use the ABQLFT
method to fit test any worker, and it may result in the selection of
[the] wrong respirator for workers with Levels 2 or 3 sensitivity * *
*" (Ex. OSHA-2007-0006-0017.1). None of these comments challenged the
validity or accuracy of the Level 1 sensitivity procedure; accordingly,
OSHA concludes that the ABQLFT study used the procedure appropriately,
and that it accurately screened the test subjects for sensitivity to
Bitrex®.
7. Miscellaneous issues addressed by the comments. Several
commenters objected that the test subjects in the ABQLFT study did not
perform seal checks while using the respirators. For example, James S.
Johnson stated that "[t]he exclusion of the users seal check may bias
the data and this isn't representative of how this procedure is
normally done" (Ex. OSHA-2007-0006-0028). In response to the
commenters, OSHA notes that the test subjects in the study used
respirators that were one or two sizes too small or too large to ensure
that a number of poor respirator fits occurred. This procedure induced
poor facepiece-to-face seals, which caused the respirators to leak.
These leaks, in turn, provided data for use in determining how
effectively the proposed ABQLFT protocol detected such leaks. The
authors of the ABQLFT study explained the absence of seal checks as
follows: "Experience in this laboratory has shown that people who
participate in fit tests on a frequent basis and who are allowed to
perform user seal checks can adjust most respirators to fit well enough
to pass a fit test (Janssen, 2002). For this reason, the subjects were
instructed to adjust the facepiece until comfortable but were not
permitted to perform a user seal check" (Ex. OSHA-2007-0006-0003).
Therefore, OSHA concludes that removing seal checks from the study was
necessary to obtain leakage data for use in determining the
effectiveness of the proposed ABQLFT protocol.
D. Conclusions
Based on a complete and thorough review of the rulemaking record,
OSHA concludes that:
1. The study was well controlled, and conducted according to
accepted experimental design practices and principles.
2. The authors of the studies described in the submitted articles
presented the the results properly, fully, and fairly in the context of
the ANSI Z88.10-2001 consensus standard.
3. The results generated by the proposed protocol provided
reproducible fit-testing results, and the experiments and analyses were
adequate for this purpose.
4. The results for the proposed protocol were reliable, but OSHA
can reach no conclusion regarding how the proposed protocol compares to
other qualitative fit-testing protocols because the study did not make
these comparisons. Additionally, the study did not demonstrate that the
proposed protocol accurately determined fit for N95 filtering-facepiece
respirators; therefore, OSHA could not approve the proposed protocol
for fit testing this class of respirators.
5. The test-sensitivity value of 0.92 would increase substantially
the number of employees who would pass the proposed protocol with
improperly fitting respirators, thereby making the proposed protocol
unacceptable for listing in Part I.B. of Appendix A of OSHA's
Respiratory Protection Standard. In addition, using binary logistic-
regression analysis as a substitute for the sensitivity-test criterion
in ANSI Z88.10-2001 is premature because the analysis requires
additional validation.
6. The results indicate that limiting the proposed protocol to test
subjects who demonstrated Level 1 sensitivity to Bitrex® was
appropriate.
7. To ensure adequate respirator leakage, the study justifiably
omitted seal checks from the experimental procedures.
Additional validation testing of, or revisions to, the proposed
ABQLFT protocol may provide new results for the protocol that meet or
exceed the sensitivity-test criterion established by the ANSI Z88.10
consensus standard. After submitting these new results and supporting
documentation to OSHA, OSHA would evaluate this information and, if
appropriate, would submit it to the public for notice and comment. If
the revised protocol is to apply to filtering-facepiece respirators,
then the resubmission should include testing on these respirators
demonstrating that the revised protocol accurately identifies poor fit
among test subjects who use them.
List of Subjects in 29 CFR Part 1910
Hazardous substances, Health, Occupational safety and health, Toxic
substances.
Authority and Signature
Jordan Barab, Acting Assistant Secretary of Labor for Occupational
Safety and Health, U.S. Department of Labor, 200 Constitution Avenue,
NW., Washington, DC 20210, directed the preparation of this notice.
Accordingly, the Agency issues this notice under the following
authorities: Sections 4, 6(b), 8(c), and 8(g) of the Occupational
Safety and Health Act of 1970 (29 U.S.C. 653, 655, 657); Section 3704
of the Contract Work Hours and Safety Standards Act (40 U.S.C. 3701 et
seq.); Section 41 of the Longshore and Harbor Worker's Compensation Act
(33 U.S.C. 941); Secretary of Labor's Order No. 5-2007 (72 FR 31160);
and 29 CFR part 1911.
Signed at Washington, DC, on June 22, 2009.
Jordan Barab,
Acting Assistant Secretary of Labor for Occupational Safety and Health.
[FR Doc. E9-14979 Filed 6-24-09; 8:45 am]
BILLING CODE 4510-26-P
