Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 627b7962 authored by Takashi Iwai's avatar Takashi Iwai
Browse files

Merge branch 'topic/misc' into for-linus

parents 29abceb6 8c3f5d8a
Loading
Loading
Loading
Loading
+1 −1
Original line number Diff line number Diff line
@@ -404,7 +404,7 @@
  /* SNDRV_CARDS: maximum number of cards supported by this module */
  static int index[SNDRV_CARDS] = SNDRV_DEFAULT_IDX;
  static char *id[SNDRV_CARDS] = SNDRV_DEFAULT_STR;
  static int enable[SNDRV_CARDS] = SNDRV_DEFAULT_ENABLE_PNP;
  static bool enable[SNDRV_CARDS] = SNDRV_DEFAULT_ENABLE_PNP;

  /* definition of the chip-specific record */
  struct mychip {
+188 −0
Original line number Diff line number Diff line
		compress_offload.txt
		=====================
	Pierre-Louis.Bossart <pierre-louis.bossart@linux.intel.com>
		Vinod Koul <vinod.koul@linux.intel.com>

Overview

Since its early days, the ALSA API was defined with PCM support or
constant bitrates payloads such as IEC61937 in mind. Arguments and
returned values in frames are the norm, making it a challenge to
extend the existing API to compressed data streams.

In recent years, audio digital signal processors (DSP) were integrated
in system-on-chip designs, and DSPs are also integrated in audio
codecs. Processing compressed data on such DSPs results in a dramatic
reduction of power consumption compared to host-based
processing. Support for such hardware has not been very good in Linux,
mostly because of a lack of a generic API available in the mainline
kernel.

Rather than requiring a compability break with an API change of the
ALSA PCM interface, a new 'Compressed Data' API is introduced to
provide a control and data-streaming interface for audio DSPs.

The design of this API was inspired by the 2-year experience with the
Intel Moorestown SOC, with many corrections required to upstream the
API in the mainline kernel instead of the staging tree and make it
usable by others.

Requirements

The main requirements are:

- separation between byte counts and time. Compressed formats may have
  a header per file, per frame, or no header at all. The payload size
  may vary from frame-to-frame. As a result, it is not possible to
  estimate reliably the duration of audio buffers when handling
  compressed data. Dedicated mechanisms are required to allow for
  reliable audio-video synchronization, which requires precise
  reporting of the number of samples rendered at any given time.

- Handling of multiple formats. PCM data only requires a specification
  of the sampling rate, number of channels and bits per sample. In
  contrast, compressed data comes in a variety of formats. Audio DSPs
  may also provide support for a limited number of audio encoders and
  decoders embedded in firmware, or may support more choices through
  dynamic download of libraries.

- Focus on main formats. This API provides support for the most
  popular formats used for audio and video capture and playback. It is
  likely that as audio compression technology advances, new formats
  will be added.

- Handling of multiple configurations. Even for a given format like
  AAC, some implementations may support AAC multichannel but HE-AAC
  stereo. Likewise WMA10 level M3 may require too much memory and cpu
  cycles. The new API needs to provide a generic way of listing these
  formats.

- Rendering/Grabbing only. This API does not provide any means of
  hardware acceleration, where PCM samples are provided back to
  user-space for additional processing. This API focuses instead on
  streaming compressed data to a DSP, with the assumption that the
  decoded samples are routed to a physical output or logical back-end.

 - Complexity hiding. Existing user-space multimedia frameworks all
  have existing enums/structures for each compressed format. This new
  API assumes the existence of a platform-specific compatibility layer
  to expose, translate and make use of the capabilities of the audio
  DSP, eg. Android HAL or PulseAudio sinks. By construction, regular
  applications are not supposed to make use of this API.


Design

The new API shares a number of concepts with with the PCM API for flow
control. Start, pause, resume, drain and stop commands have the same
semantics no matter what the content is.

The concept of memory ring buffer divided in a set of fragments is
borrowed from the ALSA PCM API. However, only sizes in bytes can be
specified.

Seeks/trick modes are assumed to be handled by the host.

The notion of rewinds/forwards is not supported. Data committed to the
ring buffer cannot be invalidated, except when dropping all buffers.

The Compressed Data API does not make any assumptions on how the data
is transmitted to the audio DSP. DMA transfers from main memory to an
embedded audio cluster or to a SPI interface for external DSPs are
possible. As in the ALSA PCM case, a core set of routines is exposed;
each driver implementer will have to write support for a set of
mandatory routines and possibly make use of optional ones.

The main additions are

- get_caps
This routine returns the list of audio formats supported. Querying the
codecs on a capture stream will return encoders, decoders will be
listed for playback streams.

- get_codec_caps For each codec, this routine returns a list of
capabilities. The intent is to make sure all the capabilities
correspond to valid settings, and to minimize the risks of
configuration failures. For example, for a complex codec such as AAC,
the number of channels supported may depend on a specific profile. If
the capabilities were exposed with a single descriptor, it may happen
that a specific combination of profiles/channels/formats may not be
supported. Likewise, embedded DSPs have limited memory and cpu cycles,
it is likely that some implementations make the list of capabilities
dynamic and dependent on existing workloads. In addition to codec
settings, this routine returns the minimum buffer size handled by the
implementation. This information can be a function of the DMA buffer
sizes, the number of bytes required to synchronize, etc, and can be
used by userspace to define how much needs to be written in the ring
buffer before playback can start.

- set_params
This routine sets the configuration chosen for a specific codec. The
most important field in the parameters is the codec type; in most
cases decoders will ignore other fields, while encoders will strictly
comply to the settings

- get_params
This routines returns the actual settings used by the DSP. Changes to
the settings should remain the exception.

- get_timestamp
The timestamp becomes a multiple field structure. It lists the number
of bytes transferred, the number of samples processed and the number
of samples rendered/grabbed. All these values can be used to determine
the avarage bitrate, figure out if the ring buffer needs to be
refilled or the delay due to decoding/encoding/io on the DSP.

Note that the list of codecs/profiles/modes was derived from the
OpenMAX AL specification instead of reinventing the wheel.
Modifications include:
- Addition of FLAC and IEC formats
- Merge of encoder/decoder capabilities
- Profiles/modes listed as bitmasks to make descriptors more compact
- Addition of set_params for decoders (missing in OpenMAX AL)
- Addition of AMR/AMR-WB encoding modes (missing in OpenMAX AL)
- Addition of format information for WMA
- Addition of encoding options when required (derived from OpenMAX IL)
- Addition of rateControlSupported (missing in OpenMAX AL)

Not supported:

- Support for VoIP/circuit-switched calls is not the target of this
  API. Support for dynamic bit-rate changes would require a tight
  coupling between the DSP and the host stack, limiting power savings.

- Packet-loss concealment is not supported. This would require an
  additional interface to let the decoder synthesize data when frames
  are lost during transmission. This may be added in the future.

- Volume control/routing is not handled by this API. Devices exposing a
  compressed data interface will be considered as regular ALSA devices;
  volume changes and routing information will be provided with regular
  ALSA kcontrols.

- Embedded audio effects. Such effects should be enabled in the same
  manner, no matter if the input was PCM or compressed.

- multichannel IEC encoding. Unclear if this is required.

- Encoding/decoding acceleration is not supported as mentioned
  above. It is possible to route the output of a decoder to a capture
  stream, or even implement transcoding capabilities. This routing
  would be enabled with ALSA kcontrols.

- Audio policy/resource management. This API does not provide any
  hooks to query the utilization of the audio DSP, nor any premption
  mechanisms.

- No notion of underun/overrun. Since the bytes written are compressed
  in nature and data written/read doesn't translate directly to
  rendered output in time, this does not deal with underrun/overun and
  maybe dealt in user-library

Credits:
- Mark Brown and Liam Girdwood for discussions on the need for this API
- Harsha Priya for her work on intel_sst compressed API
- Rakesh Ughreja for valuable feedback
- Sing Nallasellan, Sikkandar Madar and Prasanna Samaga for
  demonstrating and quantifying the benefits of audio offload on a
  real platform.
+2 −0
Original line number Diff line number Diff line
@@ -6,3 +6,5 @@ header-y += hdsp.h
header-y += hdspm.h
header-y += sb16_csp.h
header-y += sfnt_info.h
header-y += compress_params.h
header-y += compress_offload.h
+167 −0
Original line number Diff line number Diff line
/*
 *  compress_driver.h - compress offload driver definations
 *
 *  Copyright (C) 2011 Intel Corporation
 *  Authors:	Vinod Koul <vinod.koul@linux.intel.com>
 *		Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
 *  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; version 2 of the License.
 *
 *  This program is distributed in the hope that it will be useful, but
 *  WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 *  General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License along
 *  with this program; if not, write to the Free Software Foundation, Inc.,
 *  59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
 *
 * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 */
#ifndef __COMPRESS_DRIVER_H
#define __COMPRESS_DRIVER_H

#include <linux/types.h>
#include <linux/sched.h>
#include <sound/compress_offload.h>
#include <sound/asound.h>
#include <sound/pcm.h>

struct snd_compr_ops;

/**
 * struct snd_compr_runtime: runtime stream description
 * @state: stream state
 * @ops: pointer to DSP callbacks
 * @buffer: pointer to kernel buffer, valid only when not in mmap mode or
 *	DSP doesn't implement copy
 * @buffer_size: size of the above buffer
 * @fragment_size: size of buffer fragment in bytes
 * @fragments: number of such fragments
 * @hw_pointer: offset of last location in buffer where DSP copied data
 * @app_pointer: offset of last location in buffer where app wrote data
 * @total_bytes_available: cumulative number of bytes made available in
 *	the ring buffer
 * @total_bytes_transferred: cumulative bytes transferred by offload DSP
 * @sleep: poll sleep
 */
struct snd_compr_runtime {
	snd_pcm_state_t state;
	struct snd_compr_ops *ops;
	void *buffer;
	u64 buffer_size;
	u32 fragment_size;
	u32 fragments;
	u64 hw_pointer;
	u64 app_pointer;
	u64 total_bytes_available;
	u64 total_bytes_transferred;
	wait_queue_head_t sleep;
};

/**
 * struct snd_compr_stream: compressed stream
 * @name: device name
 * @ops: pointer to DSP callbacks
 * @runtime: pointer to runtime structure
 * @device: device pointer
 * @direction: stream direction, playback/recording
 * @private_data: pointer to DSP private data
 */
struct snd_compr_stream {
	const char *name;
	struct snd_compr_ops *ops;
	struct snd_compr_runtime *runtime;
	struct snd_compr *device;
	enum snd_compr_direction direction;
	void *private_data;
};

/**
 * struct snd_compr_ops: compressed path DSP operations
 * @open: Open the compressed stream
 * This callback is mandatory and shall keep dsp ready to receive the stream
 * parameter
 * @free: Close the compressed stream, mandatory
 * @set_params: Sets the compressed stream parameters, mandatory
 * This can be called in during stream creation only to set codec params
 * and the stream properties
 * @get_params: retrieve the codec parameters, mandatory
 * @trigger: Trigger operations like start, pause, resume, drain, stop.
 * This callback is mandatory
 * @pointer: Retrieve current h/w pointer information. Mandatory
 * @copy: Copy the compressed data to/from userspace, Optional
 * Can't be implemented if DSP supports mmap
 * @mmap: DSP mmap method to mmap DSP memory
 * @ack: Ack for DSP when data is written to audio buffer, Optional
 * Not valid if copy is implemented
 * @get_caps: Retrieve DSP capabilities, mandatory
 * @get_codec_caps: Retrieve capabilities for a specific codec, mandatory
 */
struct snd_compr_ops {
	int (*open)(struct snd_compr_stream *stream);
	int (*free)(struct snd_compr_stream *stream);
	int (*set_params)(struct snd_compr_stream *stream,
			struct snd_compr_params *params);
	int (*get_params)(struct snd_compr_stream *stream,
			struct snd_codec *params);
	int (*trigger)(struct snd_compr_stream *stream, int cmd);
	int (*pointer)(struct snd_compr_stream *stream,
			struct snd_compr_tstamp *tstamp);
	int (*copy)(struct snd_compr_stream *stream, const char __user *buf,
		       size_t count);
	int (*mmap)(struct snd_compr_stream *stream,
			struct vm_area_struct *vma);
	int (*ack)(struct snd_compr_stream *stream, size_t bytes);
	int (*get_caps) (struct snd_compr_stream *stream,
			struct snd_compr_caps *caps);
	int (*get_codec_caps) (struct snd_compr_stream *stream,
			struct snd_compr_codec_caps *codec);
};

/**
 * struct snd_compr: Compressed device
 * @name: DSP device name
 * @dev: Device pointer
 * @ops: pointer to DSP callbacks
 * @private_data: pointer to DSP pvt data
 * @card: sound card pointer
 * @direction: Playback or capture direction
 * @lock: device lock
 * @device: device id
 */
struct snd_compr {
	const char *name;
	struct device *dev;
	struct snd_compr_ops *ops;
	void *private_data;
	struct snd_card *card;
	unsigned int direction;
	struct mutex lock;
	int device;
};

/* compress device register APIs */
int snd_compress_register(struct snd_compr *device);
int snd_compress_deregister(struct snd_compr *device);
int snd_compress_new(struct snd_card *card, int device,
			int type, struct snd_compr *compr);

/* dsp driver callback apis
 * For playback: driver should call snd_compress_fragment_elapsed() to let the
 * framework know that a fragment has been consumed from the ring buffer
 *
 * For recording: we want to know when a frame is available or when
 * at least one frame is available so snd_compress_frame_elapsed()
 * callback should be called when a encodeded frame is available
 */
static inline void snd_compr_fragment_elapsed(struct snd_compr_stream *stream)
{
	wake_up(&stream->runtime->sleep);
}

#endif
+161 −0
Original line number Diff line number Diff line
/*
 *  compress_offload.h - compress offload header definations
 *
 *  Copyright (C) 2011 Intel Corporation
 *  Authors:	Vinod Koul <vinod.koul@linux.intel.com>
 *		Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
 *  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 *  This program is free software; you can redistribute it and/or modify
 *  it under the terms of the GNU General Public License as published by
 *  the Free Software Foundation; version 2 of the License.
 *
 *  This program is distributed in the hope that it will be useful, but
 *  WITHOUT ANY WARRANTY; without even the implied warranty of
 *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 *  General Public License for more details.
 *
 *  You should have received a copy of the GNU General Public License along
 *  with this program; if not, write to the Free Software Foundation, Inc.,
 *  59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
 *
 * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 */
#ifndef __COMPRESS_OFFLOAD_H
#define __COMPRESS_OFFLOAD_H

#include <linux/types.h>
#include <sound/asound.h>
#include <sound/compress_params.h>


#define SNDRV_COMPRESS_VERSION SNDRV_PROTOCOL_VERSION(0, 1, 0)
/**
 * struct snd_compressed_buffer: compressed buffer
 * @fragment_size: size of buffer fragment in bytes
 * @fragments: number of such fragments
 */
struct snd_compressed_buffer {
	__u32 fragment_size;
	__u32 fragments;
};

/**
 * struct snd_compr_params: compressed stream params
 * @buffer: buffer description
 * @codec: codec parameters
 * @no_wake_mode: dont wake on fragment elapsed
 */
struct snd_compr_params {
	struct snd_compressed_buffer buffer;
	struct snd_codec codec;
	__u8 no_wake_mode;
};

/**
 * struct snd_compr_tstamp: timestamp descriptor
 * @byte_offset: Byte offset in ring buffer to DSP
 * @copied_total: Total number of bytes copied from/to ring buffer to/by DSP
 * @pcm_frames: Frames decoded or encoded by DSP. This field will evolve by
 *	large steps and should only be used to monitor encoding/decoding
 *	progress. It shall not be used for timing estimates.
 * @pcm_io_frames: Frames rendered or received by DSP into a mixer or an audio
 * output/input. This field should be used for A/V sync or time estimates.
 * @sampling_rate: sampling rate of audio
 */
struct snd_compr_tstamp {
	__u32 byte_offset;
	__u32 copied_total;
	snd_pcm_uframes_t pcm_frames;
	snd_pcm_uframes_t pcm_io_frames;
	__u32 sampling_rate;
};

/**
 * struct snd_compr_avail: avail descriptor
 * @avail: Number of bytes available in ring buffer for writing/reading
 * @tstamp: timestamp infomation
 */
struct snd_compr_avail {
	__u64 avail;
	struct snd_compr_tstamp tstamp;
};

enum snd_compr_direction {
	SND_COMPRESS_PLAYBACK = 0,
	SND_COMPRESS_CAPTURE
};

/**
 * struct snd_compr_caps: caps descriptor
 * @codecs: pointer to array of codecs
 * @direction: direction supported. Of type snd_compr_direction
 * @min_fragment_size: minimum fragment supported by DSP
 * @max_fragment_size: maximum fragment supported by DSP
 * @min_fragments: min fragments supported by DSP
 * @max_fragments: max fragments supported by DSP
 * @num_codecs: number of codecs supported
 * @reserved: reserved field
 */
struct snd_compr_caps {
	__u32 num_codecs;
	__u32 direction;
	__u32 min_fragment_size;
	__u32 max_fragment_size;
	__u32 min_fragments;
	__u32 max_fragments;
	__u32 codecs[MAX_NUM_CODECS];
	__u32 reserved[11];
};

/**
 * struct snd_compr_codec_caps: query capability of codec
 * @codec: codec for which capability is queried
 * @num_descriptors: number of codec descriptors
 * @descriptor: array of codec capability descriptor
 */
struct snd_compr_codec_caps {
	__u32 codec;
	__u32 num_descriptors;
	struct snd_codec_desc descriptor[MAX_NUM_CODEC_DESCRIPTORS];
};

/**
 * compress path ioctl definitions
 * SNDRV_COMPRESS_GET_CAPS: Query capability of DSP
 * SNDRV_COMPRESS_GET_CODEC_CAPS: Query capability of a codec
 * SNDRV_COMPRESS_SET_PARAMS: Set codec and stream parameters
 * Note: only codec params can be changed runtime and stream params cant be
 * SNDRV_COMPRESS_GET_PARAMS: Query codec params
 * SNDRV_COMPRESS_TSTAMP: get the current timestamp value
 * SNDRV_COMPRESS_AVAIL: get the current buffer avail value.
 * This also queries the tstamp properties
 * SNDRV_COMPRESS_PAUSE: Pause the running stream
 * SNDRV_COMPRESS_RESUME: resume a paused stream
 * SNDRV_COMPRESS_START: Start a stream
 * SNDRV_COMPRESS_STOP: stop a running stream, discarding ring buffer content
 * and the buffers currently with DSP
 * SNDRV_COMPRESS_DRAIN: Play till end of buffers and stop after that
 * SNDRV_COMPRESS_IOCTL_VERSION: Query the API version
 */
#define SNDRV_COMPRESS_IOCTL_VERSION	_IOR('C', 0x00, int)
#define SNDRV_COMPRESS_GET_CAPS		_IOWR('C', 0x10, struct snd_compr_caps)
#define SNDRV_COMPRESS_GET_CODEC_CAPS	_IOWR('C', 0x11,\
						struct snd_compr_codec_caps)
#define SNDRV_COMPRESS_SET_PARAMS	_IOW('C', 0x12, struct snd_compr_params)
#define SNDRV_COMPRESS_GET_PARAMS	_IOR('C', 0x13, struct snd_codec)
#define SNDRV_COMPRESS_TSTAMP		_IOR('C', 0x20, struct snd_compr_tstamp)
#define SNDRV_COMPRESS_AVAIL		_IOR('C', 0x21, struct snd_compr_avail)
#define SNDRV_COMPRESS_PAUSE		_IO('C', 0x30)
#define SNDRV_COMPRESS_RESUME		_IO('C', 0x31)
#define SNDRV_COMPRESS_START		_IO('C', 0x32)
#define SNDRV_COMPRESS_STOP		_IO('C', 0x33)
#define SNDRV_COMPRESS_DRAIN		_IO('C', 0x34)
/*
 * TODO
 * 1. add mmap support
 *
 */
#define SND_COMPR_TRIGGER_DRAIN 7 /*FIXME move this to pcm.h */
#endif
Loading