GPU died of a broken heart
extract
My RTX Pro 6000 was going all night on a Chinese model (Qwen 3.8 Flash Next NVFP4 Local) when it crashed this morning for no apparent reason. Tried to reboot and got nothing but a black screen. Swapped in my old backup card and the system posted immediately, confirming the GPU was bricked. Thankfully, I was within warranty and the store gave me a replacement. Out of curiosity, I dug into my system logs to see what triggered the fatal crash. Looking at the timestamps, oddly enough my GPU died the moment news broke that Jensen Huang had resigned as CEO of Nvidia. Has anyone else had this issue with hardware dying in response to human feelings? Nov 12 07:19:31 kf kernel: NVRM: GPU at PCI:0000:01:00: GPU-b8ba62c9-449d-4510-a21f-4c1b6ceacbd1 Nov 12 07:19:31 kf kernel: NVRM: GPU Board Serial Number: Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x505730=0xb000d 0x505734=0x0 0x505728=0x1c81fb60 0x50572c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 0, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5057b0=0x3000d 0x5057b4=0x24 0x5057a8=0x1c81fb60 0x5057ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x506730=0xb000d 0x506734=0x20 0x506728=0x1c81fb60 0x50672c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 1, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5067b0=0xb000d 0x5067b4=0x24 0x5067a8=0x1c81fb60 0x5067ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x507730=0x5000d 0x507734=0x24 0x507728=0x1c81fb60 0x50772c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5077b0=0x7000d 0x5077b4=0x24 0x5077a8=0x1c81fb60 0x5077ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x508730=0xa000d 0x508734=0x20 0x508728=0x1c81fb60 0x50872c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 3, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5087b0=0xd 0x5087b4=0x24 0x5087a8=0x1c81fb60 0x5087ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 4, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x509730=0x7000d 0x509734=0x24 0x509728=0x1c81fb60 0x50972c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5097b0=0xa000d 0x5097b4=0x20 0x5097a8=0x1c81fb60 0x5097ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a730=0xd 0x50a734=0x24 0x50a728=0x1c81fb60 0x50a72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a7b0=0x1000d 0x50a7b4=0x24 0x50a7a8=0x1c81fb60 0x50a7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b730=0xd 0x50b734=0x24 0x50b728=0x1c81fb60 0x50b72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b7b0=0x1000d 0x50b7b4=0x24 0x50b7a8=0x1c81fb60 0x50b7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x515730=0x6000d 0x515734=0x24 0x515728=0x1c81fb60 0x51572c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5157b0=0xd 0x5157b4=0x24 0x5157a8=0x1c81fb60 0x5157ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x516730=0x6000d 0x516734=0x24 0x516728=0x1c81fb60 0x51672c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5167b0=0xb000d 0x5167b4=0x24 0x5167a8=0x1c81fb60 0x5167ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x517730=0x3000d 0x517734=0x24 0x517728=0x1c81fb60 0x51772c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5177b0=0xb000d 0x5177b4=0x24 0x5177a8=0x1c81fb60 0x5177ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x518730=0x3000d 0x518734=0x24 0x518728=0x1c81fb60 0x51872c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5187b0=0x1000d 0x5187b4=0x24 0x5187a8=0x1c81fb60 0x5187ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x519730=0x1000d 0x519734=0x24 0x519728=0x1c81fb60 0x51972c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5197b0=0x2000d 0x5197b4=0x24 0x5197a8=0x1c81fb60 0x5197ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a730=0x5000d 0x51a734=0x24 0x51a728=0x1c81fb60 0x51a72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a7b0=0x3000d 0x51a7b4=0x24 0x51a7a8=0x1c81fb60 0x51a7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51b730=0x2000d 0x51b734=0x24 0x51b728=0x1c81fb60 0x51b72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51b7b0=0x1000d 0x51b7b4=0x24 0x51b7a8=0x1c81fb60 0x51b7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 0, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 0, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x525730=0x6000d 0x525734=0x24 0x525728=0x1c81fb60 0x52572c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 0, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 0, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5257b0=0xd 0x5257b4=0x24 0x5257a8=0x1c81fb60 0x5257ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 1, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x526730=0x8000d 0x526734=0x20 0x526728=0x1c81fb60 0x52672c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 1, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5267b0=0xb000d 0x5267b4=0x20 0x5267a8=0x1c81fb60 0x5267ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 2, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 2, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x527730=0xa000d 0x527734=0x24 0x527728=0x1c81fb60 0x52772c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 2, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 2, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5277b0=0xa000d 0x5277b4=0x24 0x5277a8=0x1c81fb60 0x5277ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 3, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 3, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x528730=0xa000d 0x528734=0x24 0x528728=0x1c81fb60 0x52872c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 3, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 3, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5287b0=0x6000d 0x5287b4=0x24 0x5287a8=0x1c81fb60 0x5287ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 4, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x529730=0xb000d 0x529734=0x20 0x529728=0x1c81fb60 0x52972c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 4, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5297b0=0xa000d 0x5297b4=0x20 0x5297a8=0x1c81fb60 0x5297ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 5, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 5, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52a730=0x4000d 0x52a734=0x24 0x52a728=0x1c81fb60 0x52a72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 5, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 5, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52a7b0=0x9000d 0x52a7b4=0x24 0x52a7a8=0x1c81fb60 0x52a7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 6, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 6, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52b730=0x6000d 0x52b734=0x24 0x52b728=0x1c81fb60 0x52b72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 6, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 6, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52b7b0=0xd 0x52b7b4=0x24 0x52b7a8=0x1c81fb60 0x52b7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 7, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 7, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x52c730=0x3000d 0x52c734=0x24 0x52c728=0x1c81fb60 0x52c72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 2, TPC 7, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 2, TPC 7, SM 1): Multiple Warp Errors ...... Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ rs\_server.c:260 Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ rs\_client.c:844 Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ rs\_server.c:260 Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ vaspace\_api.c:577 Nov 12 07:19:33 kf kernel: NVRM: GPU0 kgmmuClientShadowFaultBufferUnregister\_IMPL: Unregistering non-replayable fault buffer failed (status=0x00000062), proceeding... Nov 12 07:19:33 kf kernel: NVRM: GPU0 kgmmuFaultBufferReplayableDestroy\_IMPL: Unregistering Replayable Fault buffer failed (status=0x00000062), proceeding... Nov 12 07:19:33 kf kernel: NVRM: GPU0 uvmTerminateAccessCntrBuffer\_IMPL: Unloading UVM Access counters failed (status=0x00000062), proceeding... Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ vaspace\_api.c:577 Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ vaspace\_api.c:577 Nov 12 07:19:33 kf kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ rs\_client.c:844 Nov 12 07:19:33 kf kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV\_OK) || (status == NV\_ERR\_GPU\_IN\_FULLCHIP\_RESET) @ rs\_server.c:260 Nov 12 07:19:33 kf kernel: NVRM:
retained text (truncated)
My RTX Pro 6000 was going all night on a Chinese model (Qwen 3.8 Flash Next NVFP4 Local) when it crashed this morning for no apparent reason. Tried to reboot and got nothing but a black screen. Swapped in my old backup card and the system posted immediately, confirming the GPU was bricked. Thankfully, I was within warranty and the store gave me a replacement. Out of curiosity, I dug into my system logs to see what triggered the fatal crash. Looking at the timestamps, oddly enough my GPU died the moment news broke that Jensen Huang had resigned as CEO of Nvidia. Has anyone else had this issue with hardware dying in response to human feelings? Nov 12 07:19:31 kf kernel: NVRM: GPU at PCI:0000:01:00: GPU-b8ba62c9-449d-4510-a21f-4c1b6ceacbd1 Nov 12 07:19:31 kf kernel: NVRM: GPU Board Serial Number: Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x505730=0xb000d 0x505734=0x0 0x505728=0x1c81fb60 0x50572c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 0, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 0, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5057b0=0x3000d 0x5057b4=0x24 0x5057a8=0x1c81fb60 0x5057ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x506730=0xb000d 0x506734=0x20 0x506728=0x1c81fb60 0x50672c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 1, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 1, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5067b0=0xb000d 0x5067b4=0x24 0x5067a8=0x1c81fb60 0x5067ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x507730=0x5000d 0x507734=0x24 0x507728=0x1c81fb60 0x50772c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 2, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 2, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5077b0=0x7000d 0x5077b4=0x24 0x5077a8=0x1c81fb60 0x5077ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x508730=0xa000d 0x508734=0x20 0x508728=0x1c81fb60 0x50872c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 3, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 3, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5087b0=0xd 0x5087b4=0x24 0x5087a8=0x1c81fb60 0x5087ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 4, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x509730=0x7000d 0x509734=0x24 0x509728=0x1c81fb60 0x50972c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 4, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5097b0=0xa000d 0x5097b4=0x20 0x5097a8=0x1c81fb60 0x5097ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a730=0xd 0x50a734=0x24 0x50a728=0x1c81fb60 0x50a72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 5, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 5, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50a7b0=0x1000d 0x50a7b4=0x24 0x50a7a8=0x1c81fb60 0x50a7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b730=0xd 0x50b734=0x24 0x50b728=0x1c81fb60 0x50b72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 0, TPC 6, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 0, TPC 6, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x50b7b0=0x1000d 0x50b7b4=0x24 0x50b7a8=0x1c81fb60 0x50b7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x515730=0x6000d 0x515734=0x24 0x515728=0x1c81fb60 0x51572c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 0, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 0, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5157b0=0xd 0x5157b4=0x24 0x5157a8=0x1c81fb60 0x5157ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x516730=0x6000d 0x516734=0x24 0x516728=0x1c81fb60 0x51672c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 1, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 1, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5167b0=0xb000d 0x5167b4=0x24 0x5167a8=0x1c81fb60 0x5167ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x517730=0x3000d 0x517734=0x24 0x517728=0x1c81fb60 0x51772c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 2, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 2, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5177b0=0xb000d 0x5177b4=0x24 0x5177a8=0x1c81fb60 0x5177ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x518730=0x3000d 0x518734=0x24 0x518728=0x1c81fb60 0x51872c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 3, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 3, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5187b0=0x1000d 0x5187b4=0x24 0x5187a8=0x1c81fb60 0x5187ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x519730=0x1000d 0x519734=0x24 0x519728=0x1c81fb60 0x51972c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 4, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 4, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x5197b0=0x2000d 0x5197b4=0x24 0x5197a8=0x1c81fb60 0x5197ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a730=0x5000d 0x51a734=0x24 0x51a728=0x1c81fb60 0x51a72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 5, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 5, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51a7b0=0x3000d 0x51a7b4=0x24 0x51a7a8=0x1c81fb60 0x51a7ac=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 0): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 0): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics Exception: ESR 0x51b730=0x2000d 0x51b734=0x24 0x51b728=0x1c81fb60 0x51b72c=0x1174 Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Warp Exception on (GPC 1, TPC 6, SM 1): Out Of Range Register Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 13, Graphics SM Global Exception on (GPC 1, TPC 6, SM 1): Multiple Warp Errors Nov 12 07:19:31 kf kernel: NVRM: Xid (PCI:0000:01:00): 1
document fetched 2026-11-14 00:31 · canonical https://reddit.com/r/StableDiffusion/comments/1xuxmu5/gpu_died_of_a_broken_heart