windbg 디버깅 사례: AppDomain 간의 static 변수 사용으로 인한 crash (2)
w3wp.exe가 초기 실행 시 비정상 종료하는 현상이 발생했습니다. 이때의 이벤트 로그를 보면 다음과 같은 4가지 오류 메시지를 볼 수 있습니다.
Log Name: Application
Source: .NET Runtime
Date: 2020-07-29 오후 10:42:56
Event ID: 1023
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: testpc.testad.com
Description:
.NET Runtime version 2.0.50727.8806 - Fatal Execution Engine Error (000007FEF82D6A0E) (80131506)
Log Name: Application
Source: Application Error
Date: 2020-07-29 오후 10:42:56
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: testpc.testad.com
Description:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: mscorwks.dll, version: 2.0.50727.8806, time stamp: 0x5c9c4c44
Exception code: 0xc0000005
Fault offset: 0x00000000001afbb5
Faulting process id: 0x%9
Faulting application start time: 0x%10
Faulting application path: %11
Faulting module path: %12
Report Id: %13
Log Name: Application
Source: Windows Error Reporting
Date: 2020-07-29 오후 10:42:56
Event ID: 1001
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: testpc.testad.com
Description:
Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: w3wp.exe
P2: 7.5.7601.17514
P3: 4ce7afa2
P4: mscorwks.dll
P5: 2.0.50727.8806
P6: 5c9c4c44
P7: c0000005
P8: 00000000001afbb5
P9:
P10:
...[생략]...
Log Name: Application
Source: Windows Error Reporting
Date: 2020-07-29 오후 10:43:03
Event ID: 1001
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: testpc.testad.com
Description:
Fault bucket 1963194714, type 317853907
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: w3wp.exe
P2: 7.5.7601.17514
P3: 4ce7afa2
P4: mscorwks.dll
P5: 2.0.50727.8806
P6: 5c9c4c44
P7: c0000005
P8: 00000000001afbb5
P9:
P10:
...[생략]...
Watson bucket 정보가 포함되어 있지만 사용자 코드가 아닌 다음에야 딱히 문제 분석에 도움이 안 될 확률이 높습니다. ^^;
이런 식으로 초기에 비정상 종료하는 프로세스는 전에 설명했던 방식을 이용해,
IIS 7.5 - Global.asax.cs처럼 초기에 실행되는 코드에 Breakpoint를 잡는 방법
; https://www.sysnet.pe.kr/2/0/1521
w3wp.exe에서 실행하는 코드를 최소화하도록 만든 상태에서 Visual Studio의 원격 디버깅이나 windbg를 연결하는 식으로 진행해야 합니다.
일단, 그렇게 해서 덤프를 뜬 후 windbg의 기본 명령어로 분석해 보면,
0:005> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
Failed to request MethodData, not in JIT code range
MethodDesc: 000007fe98e3b330
Method Name: System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98e41d38
MethodTable: 000007fe98e3b408
mdToken: 06004537
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7c500
MethodDesc: 000007fe98e54d78
Method Name: System.Runtime.Remoting.Messaging.ServerObjectTerminatorSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98e4ce30
MethodTable: 000007fe98e54da8
mdToken: 0600459a
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7c3b0
MethodDesc: 000007fe98e55098
Method Name: System.Runtime.Remoting.Lifetime.LeaseSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98e4d198
MethodTable: 000007fe98e550c8
mdToken: 06004005
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7c350
MethodDesc: 000007fe98ee9360
Method Name: System.Runtime.Remoting.Messaging.ServerContextTerminatorSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98ef0b40
MethodTable: 000007fe98ee9390
mdToken: 0600458f
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7c0f0
MethodDesc: 000007fe98ee9140
Method Name: System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessageCallback(System.Object[])
Class: 000007fe98ef03a8
MethodTable: 000007fe98ee91a8
mdToken: 06003eb1
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7bb30
MethodDesc: 000007fe98c043e8
Method Name: System.Threading.Thread.InternalCrossContextCallback(System.Runtime.Remoting.Contexts.Context, System.Threading.InternalCrossContextDelegate, System.Object[])
Class: 000007fe98bfdfb8
MethodTable: 000007fe98c047d8
mdToken: 06001364
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7bac0
MethodDesc: 000007fe98ee9150
Method Name: System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98ef03a8
MethodTable: 000007fe98ee91a8
mdToken: 06003eb2
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7b890
MethodDesc: 000007fe98e39408
Method Name: System.Runtime.Remoting.Channels.ChannelServices.SyncDispatchMessage(System.Runtime.Remoting.Messaging.IMessage)
Class: 000007fe98e0d8e0
MethodTable: 000007fe98e39490
mdToken: 06003dca
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e7a630
MethodDesc: 000007fe98e55360
Method Name: System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCallback(System.Object[])
Class: 000007fe98e4fb98
MethodTable: 000007fe98e553b8
mdToken: 06003ee4
Module: 000007fe98b12020
IsJitted: yes
CodeAddr: 000007fe98e78df0
KEY_VALUES_STRING: 1
Key : AV.Dereference
Value: NullClassPtr
Key : AV.Fault
Value: Read
Key : Timeline.OS.Boot.DeltaSec
Value: 4174404
Key : Timeline.Process.Start.DeltaSec
Value: 1142
PROCESSES_ANALYSIS: 1
SERVICE_ANALYSIS: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
Timeline: !analyze.Start
Name: <blank>
Time: 2020-07-29T02:22:48.130Z
Diff: 440130 mSec
Timeline: Dump.Current
Name: <blank>
Time: 2020-07-29T02:15:28.0Z
Diff: 0 mSec
Timeline: Process.Start
Name: <blank>
Time: 2020-07-29T01:56:26.0Z
Diff: 1142000 mSec
Timeline: OS.Boot
Name: <blank>
Time: 2020-06-10T18:42:04.0Z
Diff: -120563296 mSec
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=000000000000000d rbx=000007fe98eebe38 rcx=0000000000000000
rdx=000007fe98eebe38 rsi=000000000000000d rdi=0000000000000000
rip=000007fef835fbb5 rsp=0000000001a3c530 rbp=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=000007fe98eeb548
r11=000000000000004c r12=0000000000000400 r13=0000000000000000
r14=0000000000000001 r15=000007fe98eebe38
iopl=0 nv up ei pl nz ac pe cy
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00010213
mscorwks!DomainLocalModule::GetClassFlags+0x45:
000007fe`f835fbb5 0fb6443828 movzx eax,byte ptr [rax+rdi+28h] ds:00000000`00000035=??
Resetting default scope
FAULTING_IP:
mscorwks!DomainLocalModule::GetClassFlags+45
000007fe`f835fbb5 0fb6443828 movzx eax,byte ptr [rax+rdi+28h]
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 000007fef835fbb5 (mscorwks!DomainLocalModule::GetClassFlags+0x0000000000000045)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 0000000000000035
Attempt to read from address 0000000000000035
DEFAULT_BUCKET_ID: NULL_CLASS_PTR_READ
PROCESS_NAME: w3wp.exe
FOLLOWUP_IP:
mscorwks!DomainLocalModule::GetClassFlags+45
000007fe`f835fbb5 0fb6443828 movzx eax,byte ptr [rax+rdi+28h]
READ_ADDRESS: 0000000000000035
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: 0000000000000035
WATSON_BKT_PROCSTAMP: 4ce7afa2
WATSON_BKT_PROCVER: 7.5.7601.17514
PROCESS_VER_PRODUCT: Internet Information Services
WATSON_BKT_MODULE: mscorwks.dll
WATSON_BKT_MODSTAMP: 5c9c4c44
WATSON_BKT_MODOFFSET: 1afbb5
WATSON_BKT_MODVER: 2.0.50727.8806
BUILD_VERSION_STRING: 6.1.7601.24545 (win7sp1_ldr_escrow.200102-1707)
MODLIST_WITH_TSCHKSUM_HASH: 435c710d0fbb47b711fe07c0c633b41f14d442a3
MODLIST_SHA1_HASH: e2cc4a6e05f30da78f5aff96da39d0c4190a138f
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
PRODUCT_TYPE: 3
SUITE_MASK: 274
DUMP_FLAGS: c07
DUMP_TYPE: 3
MISSING_CLR_SYMBOL: 0
ANALYSIS_SESSION_HOST: testpc
ANALYSIS_SESSION_TIME: 07-29-2020 11:22:48.0130
ANALYSIS_VERSION: 10.0.18362.1 amd64fre
MANAGED_CODE: 1
MANAGED_ENGINE_MODULE: mscorwks
MANAGED_ANALYSIS_PROVIDER: SOS
MANAGED_THREAD_ID: 22fc
THREAD_ATTRIBUTES:
OS_LOCALE: KOR
BUGCHECK_STR: APPLICATION_FAULT_NULL_CLASS_PTR_READ_NULL_CLASS_PTR_DEREFERENCE_INVALID_POINTER_READ
PRIMARY_PROBLEM_CLASS: APPLICATION_FAULT
PROBLEM_CLASSES:
ID: [0n313]
Type: [@ACCESS_VIOLATION]
Class: Addendum
Scope: BUCKET_ID
Name: Omit
Data: Omit
PID: [Unspecified]
TID: [0x22fc]
Frame: [0] : mscorwks!DomainLocalModule::GetClassFlags
ID: [0n285]
Type: [INVALID_POINTER_READ]
Class: Primary
Scope: BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [0x22fc]
Frame: [0] : mscorwks!DomainLocalModule::GetClassFlags
ID: [0n309]
Type: [NULL_CLASS_PTR_DEREFERENCE]
Class: Primary
Scope: BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [0x22fc]
Frame: [0] : mscorwks!DomainLocalModule::GetClassFlags
ID: [0n310]
Type: [NULL_CLASS_PTR_READ]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [0x2a04]
TID: [0x22fc]
Frame: [0] : mscorwks!DomainLocalModule::GetClassFlags
LAST_CONTROL_TRANSFER: from 000007fef836d981 to 000007fef835fbb5
STACK_TEXT:
00000000`01a3c530 000007fe`f836d981 : 00000000`00000000 00000000`00000000 000007fe`98eebe38 00000000`00000000 : mscorwks!DomainLocalModule::GetClassFlags+0x45
00000000`01a3c560 000007fe`f87f21af : 0314cc00`0916d3e9 000007fe`000000e9 000007fe`98eebe38 000007fe`98eeb548 : mscorwks!DomainLocalModule::IsClassAllocated+0x9
00000000`01a3c590 000007fe`f87db3a2 : 000007fe`98e3b408 000007fe`98e3b3b0 00000000`00000001 00000000`00000000 : mscorwks!MethodTable::CheckRunClassInitThrowing+0x3f
00000000`01a3c5d0 000007fe`f833741a : 0000006a`00000000 00000000`0114dff0 00000000`ffb18590 000007fe`98eeb548 : mscorwks!MethodDesc::DoPrestub+0x162
00000000`01a3c6c0 000007fe`f8468867 : 00000000`ffb185a8 00000000`ffb18590 00000000`ffb1a098 00000000`02466910 : mscorwks!PreStubWorker+0x1fa
00000000`01a3c780 000007fe`98e7c53d : 00000000`ffb185a8 00000000`01a3c878 00000000`ffb1a098 00000000`00000000 : mscorwks!ThePreStubAMD64+0x87
00000000`01a3c850 000007fe`98e7c451 : 00000000`ffb185a8 000007fe`98b20738 00000000`ffb18b84 00000000`00000000 : 0x000007fe`98e7c53d
00000000`01a3c8c0 000007fe`98e7c381 : 00000000`ffb18590 00000000`ffb1a098 00000000`ffb18b84 00000000`00000000 : 0x000007fe`98e7c451
00000000`01a3c910 000007fe`98e7c201 : 00000000`ffb18ed8 00000000`ffb1a098 000007fe`98b207a0 000007fe`98c76378 : 0x000007fe`98e7c381
00000000`01a3c950 000007fe`98e7bcec : 00000000`ffb1ad28 00000000`ffb1a098 000007fe`98b20770 000007fe`98c76378 : 0x000007fe`98e7c201
00000000`01a3c9c0 000007fe`f8468982 : 00000000`ffb1aa90 00000000`ffb1aa90 00000001`ffff0000 000007fe`00000000 : 0x000007fe`98e7bcec
00000000`01a3ca60 000007fe`f8390d93 : ffffffff`fffffffe 00000000`02466910 00000000`00000001 00000000`00000000 : mscorwks!CallDescrWorker+0x82
00000000`01a3cab0 000007fe`f8390eda : 00000000`01a3cd18 00000000`02466910 000007fe`00000001 00000000`00000000 : mscorwks!CallDescrWorkerWithHandler+0xd3
00000000`01a3cb50 000007fe`f83c7b5b : 00000001`98e7ba00 00000000`000000e9 00000000`000000e9 00000000`00000000 : mscorwks!DispatchCallDebuggerWrapper+0x3e
00000000`01a3cbb0 000007fe`f88c69bb : 000007fe`98b1f860 000007fe`98e7ba00 00000000`01a3cbd0 00000001`3f933db0 : mscorwks!DispatchCallNoEH+0x5f
00000000`01a3cc30 000007fe`98e7baf8 : 00000000`01a3d498 00000000`01a3cf90 00000001`3f933db0 00000000`00000000 : mscorwks!ThreadNative::InternalCrossContextCallback+0x30b
00000000`01a3cf30 000007fe`98e7b950 : 00000001`3f933db0 00000000`ffb0d940 00000000`ffb1aa38 00000000`ffb1aa90 : 0x000007fe`98e7baf8
00000000`01a3cf90 000007fe`98e7a728 : 00000000`ffb1aa78 00000000`ffb1a098 00000000`ffb1aa1c 00000000`00000000 : 0x000007fe`98e7b950
00000000`01a3d000 000007fe`98e79043 : 00000000`ffb1a098 00000001`3f943298 00000000`794cc2ef 00000000`794cc2ef : 0x000007fe`98e7a728
00000000`01a3d080 000007fe`98e78e8a : 00000000`00000000 00000000`ffb19548 00000000`01a3d108 000007fe`f832a24c : 0x000007fe`98e79043
00000000`01a3d0e0 000007fe`f8468982 : 00000000`ffb1a008 00000000`ffb1a008 00000000`023c00d0 000007fe`f832a24c : 0x000007fe`98e78e8a
00000000`01a3d160 000007fe`f8390d93 : 00000000`01a3d2a8 00000000`00000000 00000000`023bfd60 00000000`00000000 : mscorwks!CallDescrWorker+0x82
00000000`01a3d1b0 000007fe`f8390eda : 00000000`01a3d418 00000000`000000e9 00000000`00000003 000007fe`f8353c1b : mscorwks!CallDescrWorkerWithHandler+0xd3
00000000`01a3d250 000007fe`f83c7b5b : 00000000`00000000 00000000`00000000 00000000`0570f3e0 00000000`02466910 : mscorwks!DispatchCallDebuggerWrapper+0x3e
00000000`01a3d2b0 000007fe`f88c69bb : 00000000`0570f3e0 00000000`ffb1a008 00000000`00000001 00000000`05ba10e0 : mscorwks!DispatchCallNoEH+0x5f
00000000`01a3d330 000007fe`98e78d6e : 00000000`01a3ddb8 00000000`01a3d690 00000000`ffb19548 00000000`00000003 : mscorwks!ThreadNative::InternalCrossContextCallback+0x30b
00000000`01a3d630 000007fe`98e776bc : 00000000`ffb10cd0 00000000`00000000 00000000`ffb19548 00000000`01a3d6b8 : 0x000007fe`98e78d6e
00000000`01a3d690 000007fe`98e77083 : 00000000`ffb10cd0 00000000`ffb19318 00000000`00000001 00000001`3f933d01 : 0x000007fe`98e776bc
00000000`01a3d720 000007fe`98e760d3 : 00000000`ffb10cd0 00000000`ffb19318 00000000`00000000 00000001`3f933db0 : 0x000007fe`98e77083
00000000`01a3d790 000007fe`98e750c6 : 00000000`ffa0c640 00000000`ffb19318 00000000`00000000 00000000`ffa0c640 : 0x000007fe`98e760d3
00000000`01a3d840 000007fe`f841878d : 00000000`05ba0ae0 00000000`0c000000 00000000`01a3ddb8 000007fe`f835cb3e : 0x000007fe`98e750c6
00000000`01a3d9f0 000007fe`f8688a1b : 00000000`01a3ddb8 00000000`ffb19210 00000000`01a3db50 00000000`00000001 : mscorwks!`ZapImportTable::GetClassBlob'::`2'::ZapClassImportBlob::GetScope+0xed
00000000`01a3da80 000007fe`f8468b74 : 00000000`00000000 000007fe`f8468867 00000000`00000000 00000000`01a3e0e0 : mscorwks! ?? ::FNODOBFM::`string'+0x87c2b
00000000`01a3dd40 000007fe`991e115f : 00000000`ffb19260 00000000`01a3db58 00000000`00000097 00000000`0000010c : mscorwks!TransparentProxyStub_CrossContext+0x64
00000000`01a3de20 000007fe`991e095c : 00000000`00000000 00000000`02466910 00000000`04e93790 000040bf`d59192bb : 0x000007fe`991e115f
00000000`01a3ded0 000007fe`f8468982 : 00000000`ffa0c9f8 00000000`ff9dd7e8 00000000`ff94b708 00000000`00180000 : 0x000007fe`991e095c
00000000`01a3df30 000007fe`f8390d93 : 00000001`00000000 000007fe`f7b306b2 00000000`01a3dc01 00000000`00000000 : mscorwks!CallDescrWorker+0x82
00000000`01a3df90 000007fe`f8848e21 : 00000000`01a3e0c8 00000000`00000000 00000000`01a3e2e8 00000000`00000008 : mscorwks!CallDescrWorkerWithHandler+0xd3
00000000`01a3e030 000007fe`f82d03ba : 000007fe`98c82c78 00000000`ff94b708 000007fe`98c82c78 000007fe`98c82c78 : mscorwks!MethodDesc::CallDescr+0x2b1
00000000`01a3e280 000007fe`f88bb871 : 00000000`00000005 000007fe`98e18360 000007fe`98e182a0 00000000`02466910 : mscorwks!InvokeImpl+0x40a
00000000`01a3e5d0 000007fe`990a2175 : 00000000`ff94b708 00000000`ffa0c9f8 00000000`ffb04950 00000000`01a3e820 : mscorwks!RuntimeMethodHandle::InvokeMethodFast+0x131
00000000`01a3e7b0 000007fe`990a194a : 00000000`ffa0c9f8 000007fe`98e182a0 000007fe`98e18360 ffffffff`00000005 : 0x000007fe`990a2175
00000000`01a3e880 000007fe`990a16b6 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x000007fe`990a194a
00000000`01a3e910 000007fe`99294331 : 00000000`01a3e940 00000000`00000000 00000000`00000002 00000000`ffa0c9f8 : 0x000007fe`990a16b6
00000000`01a3e960 000007fe`9929f445 : 00000000`ffa0c9f8 00000000`00000000 00000000`00000000 00000000`00000244 : 0x000007fe`99294331
00000000`01a3e9a0 000007fe`99295023 : 00000000`ffa0c9f8 00000000`ffa0b5d8 00000000`ff9f0d60 00000000`00000002 : 0x000007fe`9929f445
00000000`01a3ea30 000007fe`9928ed4b : 00000000`ffa0b5d8 00000000`ffa0c9f8 00000000`ffacccd8 00000000`ffae561e : 0x000007fe`99295023
00000000`01a3eae0 000007fe`9928e37d : 00000000`ffa0c9f8 00000000`01ef4440 00000000`ffa0b5d8 00000000`ff9f8638 : 0x000007fe`9928ed4b
00000000`01a3eba0 000007fe`9917e3f2 : 00000000`ffa0c9f8 00000000`ff9dda48 00000000`ff9f8638 00000000`01ef4440 : 0x000007fe`9928e37d
00000000`01a3ec40 000007fe`9917aee4 : 00000000`ff9dd7e8 00000000`01ef4440 00000000`ffa0b5d8 000007fe`98c09658 : 0x000007fe`9917e3f2
00000000`01a3ecd0 000007fe`f8468982 : 00000000`ffa047d8 00000000`01ef4440 00000000`00e15d18 00000000`011a6e50 : 0x000007fe`9917aee4
00000000`01a3ed90 000007fe`f8390d93 : 00000000`01a3ee60 00000000`01a3ef38 ffffffff`fffffffe 00000000`00000000 : mscorwks!CallDescrWorker+0x82
00000000`01a3ede0 000007fe`f839b65e : 00000000`01a3ef70 00000000`01a3f500 00000000`00000002 00000000`00000002 : mscorwks!CallDescrWorkerWithHandler+0xd3
00000000`01a3ee80 000007fe`f8957b36 : 000007fe`98d9a598 000007fe`00000008 000007fe`00000002 00000000`00000000 : mscorwks!ForwardCallToManagedMethod+0x172
00000000`01a3ef20 000007fe`f83d19f0 : 00000000`03bff9d0 00000000`02466910 00000000`00000000 00000000`00e165d0 : mscorwks!COMToCLRWorkerBody+0x2b6
00000000`01a3f160 000007fe`f83819f6 : 00000000`00000002 00000000`00fcffd8 00000000`01a3f550 00000000`00000000 : mscorwks!COMToCLRWorkerDebuggerWrapper+0x50
00000000`01a3f1d0 000007fe`f8468afe : 00000000`02466910 00000000`01a3f550 00000000`ffffff00 00000000`00000000 : mscorwks!COMToCLRWorker+0x366
00000000`01a3f4c0 000007fe`f61c7f7e : 00000000`00d2fda0 00000000`01ef4440 00000000`00000001 00000000`00f9fcc0 : mscorwks!GenericComCallStub+0x5e
00000000`01a3f570 000007fe`f61c8861 : 00000000`01ef4440 00000000`00fcffd8 00000000`01ef4440 00000000`00000000 : webengine!W3_MGD_APP_CONTEXT::InitializeMgdApplication+0x1e
00000000`01a3f5a0 000007fe`fa52113f : 00000000`011b1630 00000000`00000000 00000000`003cbe80 00000000`00f99370 : webengine!W3_MGD_APP_CONTEXT::OnApplicationStart+0x171
00000000`01a3f600 000007fe`fa550cfb : 00000000`011b7388 00000000`00000000 00000000`00000001 00000000`00fcffd0 : iiscore!W3_SERVER::GlobalNotify+0x8f
00000000`01a3f640 000007fe`fa535c5f : 00000000`00000000 00000000`01a3f658 00000002`00000020 00000000`00000001 : iiscore!W3_APPLICATION::ResolveModules+0x2b
00000000`01a3f670 000007fe`fa527cd8 : 00000000`011b1628 00000000`00000002 00000000`00002710 00000000`00000000 : iiscore!memcpy+0x5f1f
00000000`01a3f6e0 000007fe`fa52a468 : 00000000`00000000 000007fe`00000001 00000000`00000000 00000000`00000000 : iiscore!W3_CONTEXT::SetupStateMachinePhase2+0x1c8
00000000`01a3f7a0 000007fe`fa52ab24 : 00000000`011b7380 00000000`011b67c0 00000000`011b7380 00000000`00000000 : iiscore!W3_CONTEXT::SetupStateMachine+0x968
00000000`01a3fcb0 000007fe`fa8210d2 : 00000000`00000000 00000000`011b6760 00000000`00000000 00000000`00404600 : iiscore!W3_MAIN_CONTEXT::OnNewRequest+0x1b0
00000000`01a3fce0 000007fe`fa82109c : 00000000`00404600 00000000`00000000 000007fe`fb7c0000 00000000`00000000 : w3dt!UL_NATIVE_REQUEST::DoWork+0x126
00000000`01a3fd40 000007fe`fb7c1fba : 00000000`0022cc40 00000000`00000000 00000000`001abbe0 00000000`00000000 : w3dt!OverlappedCompletionRoutine+0x1c
00000000`01a3fd70 000007fe`fb7c2024 : 00000000`0000061a 00000000`011b6768 000007fe`fa821080 00000000`0041f5b0 : w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x7a
00000000`01a3fdc0 000007fe`fb7c20a1 : 00000000`00000000 00000000`0041f5b0 00000000`00000000 00000000`00000001 : w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x34
00000000`01a3fdf0 00000000`7796556d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : w3tp!THREAD_MANAGER::ThreadManagerThread+0x61
00000000`01a3fe20 00000000`77ac372d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`01a3fe50 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
THREAD_SHA1_HASH_MOD_FUNC: 2a133c0607c7fcb4281386aff837b8da38a0647e
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 0d1d971b9d72038a0c9c98e958d38fc55525db2b
THREAD_SHA1_HASH_MOD: 0da65bfe55f63ee4797633fdec2ec34b611a1103
FAULT_INSTR_CODE: 3844b60f
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: mscorwks!DomainLocalModule::GetClassFlags+45
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: mscorwks
IMAGE_NAME: mscorwks.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 5c9c4c44
STACK_COMMAND: ~5s ; .ecxr ; kb
FAILURE_BUCKET_ID: NULL_CLASS_PTR_READ_c0000005_mscorwks.dll!DomainLocalModule::GetClassFlags
BUCKET_ID: X64_APPLICATION_FAULT_NULL_CLASS_PTR_READ_NULL_CLASS_PTR_DEREFERENCE_INVALID_POINTER_READ_mscorwks!DomainLocalModule::GetClassFlags+45
FAILURE_EXCEPTION_CODE: c0000005
FAILURE_IMAGE_NAME: mscorwks.dll
BUCKET_ID_IMAGE_STR: mscorwks.dll
FAILURE_MODULE_NAME: mscorwks
BUCKET_ID_MODULE_STR: mscorwks
FAILURE_FUNCTION_NAME: DomainLocalModule::GetClassFlags
BUCKET_ID_FUNCTION_STR: DomainLocalModule::GetClassFlags
BUCKET_ID_OFFSET: 45
BUCKET_ID_MODTIMEDATESTAMP: 5c9c4c44
BUCKET_ID_MODCHECKSUM: 98c66c
BUCKET_ID_MODVER_STR: 2.0.50727.8806
BUCKET_ID_PREFIX_STR: X64_APPLICATION_FAULT_NULL_CLASS_PTR_READ_NULL_CLASS_PTR_DEREFERENCE_INVALID_POINTER_READ_
FAILURE_PROBLEM_CLASS: APPLICATION_FAULT
FAILURE_SYMBOL_NAME: mscorwks.dll!DomainLocalModule::GetClassFlags
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/w3wp.exe/7.5.7601.17514/4ce7afa2/mscorwks.dll/2.0.50727.8806/5c9c4c44/c0000005/001afbb5.htm?Retriage=1
TARGET_TIME: 2020-07-29T02:15:28.000Z
OSBUILD: 7601
OSSERVICEPACK: 24545
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
OSPLATFORM_TYPE: x64
OSNAME: Windows 7
OSEDITION: Windows 7 Server (Service Pack 1) Enterprise TerminalServer SingleUserTS
USER_LCID: 0
OSBUILD_TIMESTAMP: 2020-01-03 12:36:28
BUILDDATESTAMP_STR: 200102-1707
BUILDLAB_STR: win7sp1_ldr_escrow
BUILDOSVER_STR: 6.1.7601.24545
ANALYSIS_SESSION_ELAPSED_TIME: 8da8
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:null_class_ptr_read_c0000005_mscorwks.dll!domainlocalmodule::getclassflags
FAILURE_ID_HASH: {2ca78c45-9d52-86ba-243b-31742104fd08}
Followup: MachineOwner
---------
애석하게도 사용자 코드는 없고, fail이 발생한 코드는 mscorwks!DomainLocalModule::GetClassFlags라고 하므로 역시나 도움이 안 됩니다. 그리고 호출 스택을 보면,
0:007> !dumpstack
OS Thread Id: 0x2bb4 (7)
Child-SP RetAddr Call Site
00000000019bc390 000007fef836d981 mscorwks!DomainLocalModule::GetClassFlags+0x45
00000000019bc3c0 000007fef87f21af mscorwks!DomainLocalModule::IsClassAllocated+0x9
00000000019bc3f0 000007fef87db3a2 mscorwks!MethodTable::CheckRunClassInitThrowing+0x3f
00000000019bc430 000007fef833741a mscorwks!MethodDesc::DoPrestub+0x162
00000000019bc520 000007fef8468867 mscorwks!PreStubWorker+0x1fa
00000000019bc5e0 000007fe98e6c63d mscorwks!ThePreStubAMD64+0x87
00000000019bc6b0 000007fe98e6c551 mscorlib!System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0x3d
00000000019bc720 000007fe98e6c481 mscorlib!System.Runtime.Remoting.Messaging.ServerObjectTerminatorSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0xa1
00000000019bc770 000007fe98e6c301 mscorlib!System.Runtime.Remoting.Lifetime.LeaseSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0x31
00000000019bc7b0 000007fe98e6bdec mscorlib!System.Runtime.Remoting.Messaging.ServerContextTerminatorSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0x111
00000000019bc820 000007fef8468982 mscorlib!System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessageCallback(System.Object[])+0x1bc
00000000019bc8c0 000007fef8390d93 mscorwks!CallDescrWorker+0x82
00000000019bc910 000007fef8390eda mscorwks!CallDescrWorkerWithHandler+0xd3
00000000019bc9b0 000007fef83c7b5b mscorwks!DispatchCallDebuggerWrapper+0x3e
00000000019bca10 000007fef88c69bb mscorwks!DispatchCallNoEH+0x5f
00000000019bca90 000007fe98e6bbf8 mscorwks!ThreadNative::InternalCrossContextCallback+0x30b
00000000019bcd90 000007fe98e6ba50 mscorlib!System.Threading.Thread.InternalCrossContextCallback(System.Runtime.Remoting.Contexts.Context, System.Threading.InternalCrossContextDelegate, System.Object[])+0x38
00000000019bcdf0 000007fe98e6a828 mscorlib!System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0xc0
00000000019bce60 000007fe98e69143 mscorlib!System.Runtime.Remoting.Channels.ChannelServices.SyncDispatchMessage(System.Runtime.Remoting.Messaging.IMessage)+0xf8
00000000019bcee0 000007fe98e68f8a mscorlib!System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[], System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage, System.Runtime.Remoting.Messaging.SmuggledMethodReturnMessage ByRef)+0xc3
00000000019bcf40 000007fef8468982 mscorlib!System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCallback(System.Object[])+0x9a
00000000019bcfc0 000007fef8390d93 mscorwks!CallDescrWorker+0x82
00000000019bd010 000007fef8390eda mscorwks!CallDescrWorkerWithHandler+0xd3
00000000019bd0b0 000007fef83c7b5b mscorwks!DispatchCallDebuggerWrapper+0x3e
00000000019bd110 000007fef88c69bb mscorwks!DispatchCallNoEH+0x5f
00000000019bd190 000007fe98e68e6e mscorwks!ThreadNative::InternalCrossContextCallback+0x30b
00000000019bd490 000007fe98e677bc mscorlib!System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatch(Byte[], System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage, System.Runtime.Remoting.Messaging.SmuggledMethodReturnMessage ByRef)+0x9e
00000000019bd4f0 000007fe98e67183 mscorlib!System.Runtime.Remoting.Channels.CrossAppDomainSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)+0xcc
00000000019bd580 000007fe98e661d3 mscorlib!System.Runtime.Remoting.Proxies.RemotingProxy.CallProcessMessage(System.Runtime.Remoting.Messaging.IMessageSink, System.Runtime.Remoting.Messaging.IMessage, System.Runtime.Remoting.Contexts.ArrayWithSize, System.Threading.Thread, System.Runtime.Remoting.Contexts.Context, Boolean)+0xa3
00000000019bd5f0 000007fe98e651c6 mscorlib!System.Runtime.Remoting.Proxies.RemotingProxy.InternalInvoke(System.Runtime.Remoting.Messaging.IMethodCallMessage, Boolean, Int32)+0x343
00000000019bd6a0 000007fef841878d mscorlib!System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(System.Runtime.Remoting.Proxies.MessageData ByRef, Int32)+0x446
00000000019bd850 000007fef8688a1b mscorwks!`ZapImportTable::GetClassBlob'::`2'::ZapClassImportBlob::GetScope+0xed
00000000019bd8e0 000007fef8468b74 mscorwks! ?? ::FNODOBFM::`string'+0x87c2b
00000000019bdba0 000007fe991d115f mscorwks!TransparentProxyStub_CrossContext+0x64
00000000019bdc80 000007fe991d095c TestWebSite!Jennifer40.WebSiteTest.empty.CreateNewCmpMgr()+0x16f
00000000019bdd30 000007fef8468982 TestWebSite!Jennifer40.WebSiteTest.Global.Application_Start(System.Object, System.EventArgs)+0xec
00000000019bdd90 000007fef8390d93 mscorwks!CallDescrWorker+0x82
00000000019bddf0 000007fef8848e21 mscorwks!CallDescrWorkerWithHandler+0xd3
00000000019bde90 000007fef82d03ba mscorwks!MethodDesc::CallDescr+0x2b1
00000000019be0e0 000007fef88bb871 mscorwks!InvokeImpl+0x40a
00000000019be430 000007fe99092275 mscorwks!RuntimeMethodHandle::InvokeMethodFast+0x131
00000000019be610 000007fe99091a4a mscorlib!System.RuntimeMethodHandle.InvokeMethodFast(System.Object, System.Object[], System.Signature, System.Reflection.MethodAttributes, System.RuntimeTypeHandle)+0x115
00000000019be6e0 000007fe990917b6 mscorlib!System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo, Boolean)+0x26a
00000000019be770 000007fe99293c81 mscorlib!System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)+0x26
00000000019be7c0 000007fe9929ef65 mscorlib!System.Reflection.MethodBase.Invoke(System.Object, System.Object[])+0x21
00000000019be800 000007fe99294973 System_Web!System.Web.HttpApplication.ProcessSpecialRequest(System.Web.HttpContext, System.Reflection.MethodInfo, Int32, System.Object, System.EventArgs, System.Web.SessionState.HttpSessionState)+0x145
00000000019be890 000007fe9928e69b System_Web!System.Web.HttpApplicationFactory.EnsureAppStartCalledForIntegratedMode(System.Web.HttpContext, System.Web.HttpApplication)+0x113
00000000019be940 000007fe9928dccd System_Web!System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr, System.Web.HttpContext, System.Reflection.MethodInfo[])+0xdb
00000000019bea00 000007fe9916e412 System_Web!System.Web.HttpApplication.InitSpecial(System.Web.HttpApplicationState, System.Reflection.MethodInfo[], IntPtr, System.Web.HttpContext)+0xed
00000000019beaa0 000007fe9916af04 System_Web!System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr, System.Web.HttpContext)+0x122
00000000019beb30 000007fef8468982 System_Web!System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr)+0xf4
00000000019bebf0 000007fef8390d93 mscorwks!CallDescrWorker+0x82
00000000019bec40 000007fef839b65e mscorwks!CallDescrWorkerWithHandler+0xd3
00000000019bece0 000007fef8957b36 mscorwks!ForwardCallToManagedMethod+0x172
00000000019bed80 000007fef83d19f0 mscorwks!COMToCLRWorkerBody+0x2b6
00000000019befc0 000007fef83819f6 mscorwks!COMToCLRWorkerDebuggerWrapper+0x50
00000000019bf030 000007fef8468afe mscorwks!COMToCLRWorker+0x366
00000000019bf320 000007fef61c7f7e mscorwks!GenericComCallStub+0x5e
00000000019bf3d0 000007fef61c8861 webengine!W3_MGD_APP_CONTEXT::InitializeMgdApplication+0x1e
00000000019bf400 000007fef7d6113f webengine!W3_MGD_APP_CONTEXT::OnApplicationStart+0x171
00000000019bf460 000007fef7d90cfb iiscore!W3_SERVER::GlobalNotify+0x8f
00000000019bf4a0 000007fef7d75c5f iiscore!W3_APPLICATION::ResolveModules+0x2b
00000000019bf4d0 000007fef7d67cd8 iiscore!memcpy+0x5f1f
00000000019bf540 000007fef7d6a468 iiscore!W3_CONTEXT::SetupStateMachinePhase2+0x1c8
00000000019bf600 000007fef7d6ab24 iiscore!W3_CONTEXT::SetupStateMachine+0x968
00000000019bfb10 000007fefa8110d2 iiscore!W3_MAIN_CONTEXT::OnNewRequest+0x1b0
00000000019bfb40 000007fefa81109c w3dt!UL_NATIVE_REQUEST::DoWork+0x126
00000000019bfba0 000007fefb7b1fba w3dt!OverlappedCompletionRoutine+0x1c
00000000019bfbd0 000007fefb7b2024 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x7a
00000000019bfc20 000007fefb7b20a1 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x34
00000000019bfc50 000000007796556d W3TP!THREAD_MANAGER::ThreadManagerThread+0x61
00000000019bfc80 0000000077ac372d kernel32!BaseThreadInitThunk+0xd
00000000019bfcb0 0000000000000000 ntdll!RtlUserThreadStart+0x1d
오류가 발생하는 곳이,
0:007> !name2ee mscorlib!System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage
Module: 000007fe98b02020 (mscorlib.dll)
Token: 0x0000000006004537
MethodDesc: 000007fe98e2b330
Name: System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage)
JITTED Code Address: 000007fe98e6c600
-----------------------
Token: 0x0000000006004538
MethodDesc: 000007fe98e2b338
Name: System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage, Int32, Boolean)
JITTED Code Address: 000007fe98e6cdf0
SyncProcessMessage의 진입 단계이고,
0:026> !u 000007fe98e6cdf0
Normal JIT generated code
System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage, Int32, Boolean)
Begin 000007fe98e6cdf0, size 44a
>>> 000007fe`98e6cdf0 53 push rbx
000007fe`98e6cdf1 55 push rbp
000007fe`98e6cdf2 56 push rsi
000007fe`98e6cdf3 57 push rdi
000007fe`98e6cdf4 4154 push r12
000007fe`98e6cdf6 4155 push r13
000007fe`98e6cdf8 4156 push r14
000007fe`98e6cdfa 4157 push r15
000007fe`98e6cdfc 4881eca8000000 sub rsp,0A8h
000007fe`98e6ce03 488d6c2440 lea rbp,[rsp+40h]
000007fe`98e6ce08 410fb6f1 movzx esi,r9b
...[생략]...
호출 스택에
mscorwks!PreStubWorker가 있는 걸로 봐서 JIT 컴파일 중에 발생한 것임을 짐작케 합니다.
결론을 보면, 이번 문제도 지난번의 유형과 동일합니다.
windbg 디버깅 사례: AppDomain 간의 static 변수 사용으로 인한 crash
; https://www.sysnet.pe.kr/2/0/1616
세월이 지나, ^^; 무심코 static 변수를 다시 해당 클래스에 사용한 것인데요... 참 이런 문제는 정말 디버깅이 힘듭니다. ^^;
[이 글에 대해서 여러분들과 의견을 공유하고 싶습니다. 틀리거나 미흡한 부분 또는 의문 사항이 있으시면 언제든 댓글 남겨주십시오.]