IConFace: Fine-Grained Identity Conditioning for Reference-Aware Face Restoration

Niu, Axi; Zhang, Jinyang; Qing, Senyan

doi:10.48550/arXiv.2605.02814

IConFace: Fine-Grained Identity Conditioning for Reference-Aware Face Restoration

Axi Niu^* Jinyang Zhang^* Senyan Qing

Northwestern Polytechnical University

nax@nwpu.edu.cn zhangjinyang@mail.nwpu.edu.cn qingsenyan@nwpu.edu.cn

^*Equal contribution

Reference-supported detail with target-aligned structure. The references and paired GT show a forehead mark obscured in the degraded observation. IConFace recovers this cue more clearly while preserving GT-consistent periocular structure.

CodeFormer

RefSTAR

IConFace

GT

Abstract

Severe face degradation can remove person-specific evidence, making restoration underdetermined. A generative prior may recover a sharp, plausible face yet miss localized traits that persist across images of the same person. Same-identity references supply this missing evidence, while the degraded observation anchors target structure. We propose IConFace, a fine-grained identity-conditioned framework that optionally conditions restoration on up to three same-identity references. Its hybrid concat backbone retains degraded and reference observations as dense visual tokens, preserving localized reference evidence. An identity pathway provides compact multi-reference guidance, while a degraded-structure pathway injects full-field and local-residual memories to reinforce target-aligned structure. We also introduce a human-audited benchmark that measures whether persistent localized identity details survive restoration. IConFace achieves leading reference compatibility, especially under severe degradation, and the highest observed preservation rate on this benchmark. Without references, it achieves leading learned perceptual quality across five blind-restoration benchmarks. Joint reference-based and paired-target evaluations show that reference-supported identity recovery and exact target agreement are complementary.

Dense identity evidence

Spatially distributed reference tokens preserve localized evidence, complemented by norm-weighted global identity aggregation.

Structure reinforcement

An input residual and full-field/local-residual memories repeatedly anchor restoration to the degraded target observation.

Detail-centered evaluation

A human-audited benchmark evaluates whether persistent localized identity traits survive restoration.

Key Results

3 / 3

Reference-aware benchmarks

Best RefAvg identity compatibility under both independent ArcFace and MagFace evaluation.

160 / 167

Localized traits preserved

A 95.8% human-audited preservation rate, the highest observed result on PID-167.

Blind-restoration benchmarks

Leading learned perceptual quality across all five evaluated datasets when references are unavailable.

Method

IConFace keeps reference evidence spatially accessible while reinforcing target-aligned structure from the degraded observation.

IConFace framework with dense visual tokens, identity guidance, and degraded-structure memories — The noisy scene latent is concatenated with VAE-encoded degraded-image and optional reference tokens. Norm-weighted AdaFace aggregation supplies compact multi-reference identity guidance, while the degraded-structure adapter adds an input residual and full-field/local-residual memories.

Dense reference route

References remain spatially distributed visual tokens throughout restoration.

Identity guidance

Norm-weighted aggregation provides compact multi-reference conditioning.

Degraded-structure route

Same-grid residual and memory paths reinforce target-aligned structure at multiple spatial scales.

Reference-Aware Restoration

Headline identity evaluation averages output-to-reference similarities within each sample (RefAvg). ArcFace and MagFace are independent of the AdaFace encoder used during training.

Persistent Localized Identity Details

PID-167 asks whether a localized identity trait supported by the paired GT and a same-identity reference remains visible after restoration.

PID-167 preservation rates for six restoration methods — IConFace preserves 160 of 167 audited traits, giving the highest observed preservation rate.

Localized cheek-mole preservation comparison — IConFace more completely preserves the displayed three-mole cheek pattern and its spatial arrangement.

Open the extended localized-detail gallery

Extended PID-167 localized identity-detail examples — Expanded examples of moles, scars, pigmentation, and other persistent localized traits. Red boxes identify the enlarged source regions.

Across diverse ages, poses, occlusions, illumination, and degradation conditions, IConFace recovers clear eyes and coherent facial components while preserving visible occlusion boundaries and surrounding structure.

Component Ablation

Two main-paper examples illustrate how identity guidance and degraded-structure reinforcement complement the dense concat backbone. The full two-route configuration yields the strongest displayed Ref₁ similarity in both cases.

BibTeX

@article{niu2026iconface,
  title={IConFace: Fine-Grained Identity Conditioning for Reference-Aware Face Restoration},
  author={Niu, Axi and Zhang, Jinyang and Qing, Senyan},
  journal={arXiv preprint arXiv:2605.02814},
  year={2026},
  eprint={2605.02814},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  doi={10.48550/arXiv.2605.02814},
  url={https://arxiv.org/abs/2605.02814}
}

Reference-supported detail example