Styling hooks can be used to adjust lightning-accordion-section title CSS in Salesforce Lightning Web Component(LWC).

Sample code:

HTML:

<template>
    <div class="slds-box slds-theme--default">
        <lightning-accordion allow-multiple-sections-open>
            <lightning-accordion-section name="A" label="Accordion Title A">
                Testing Accordion
            </lightning-accordion-section>
        </lightning-accordion>
    </div>
</template>

JavaScript:

import { LightningElement } from 'lwc';

export default class LightningAccordion extends LightningElement {}

CSS:

:host {
    --slds-c-accordion-heading-text-color: black;
    --slds-c-accordion-heading-font-size: 1.2rem
  }

JS-meta.xml:

<?xml version="1.0" encoding="UTF-8"?>
<LightningComponentBundle xmlns="http://soap.sforce.com/2006/04/metadata">
    <apiVersion>52.0</apiVersion>
    <isExposed>true</isExposed>
    <targets>
        <target>lightning__Tab</target>
    </targets>
</LightningComponentBundle>

Output:

For additional supported Styling Hooks, check the following
https://www.lightningdesignsystem.com/components/accordion/#Styling-Hooks-Overview