堡盟GAPI和OpenCV
堡盟GAPI v2.9.0軟件示例
本操作說明將介紹如何將圖像從堡盟GAPI幀緩沖區傳輸至OpenCV，以便進一步處理。
首先，使用堡盟GAPI SDK設置堡盟相機，并將圖像采集到內存，然后將內存中的圖像傳輸至OpenCV，以便將其保存到硬盤上。圖像傳輸到OpenCV后，即可根據需要使用該視覺庫進一步處理圖像。
將圖像從堡盟GAPI傳輸至OpenCV
在堡盟GAPI緩沖區接收到圖像數據后，即可通過重載構造函數將圖像分配到OpenCV矩陣（Mat）中：
new cv::Mat::Mat(int _rows, int _cols, int _type, Void * _data, size_t _step = 0Ui64);
new Emgu.CV.Mat.Mat(int rows, int cols, Emgu.CV.CvEnum.DepthType type,
int channels, IntPtr data, int step);
堡盟GAPI內存緩沖區的數據不會被復制，但有一個指針會被傳輸到OpenCV矩陣。之后，OpenCV矩陣可通過cv::imwrite()函數儲存到磁盤中。保存后，您可以選擇刪除或重復使用OpenCV矩陣，然后重新對堡盟GAPI緩沖區進行排序，以便將其用于采集新圖像。
將堡盟GAPI緩沖區克隆、復制并轉換為OpenCV矩陣
如需獲取堡盟GAPI緩沖區中數據的副本，可以使用OpenCV的Clone()、Copy()或ConvertTo()函數。復制后，可以重新對堡盟GAPI緩沖區進行排序，復制的數據也可以通過OpenCV進一步處理。
請注意，OpenCV矩陣中的信息包括圖像的寬度、高度和類型（CV_8UC1、CV_8UC3、CV_16UC1、CV_16UC3），而堡盟GAPI緩沖區提供Buffer.FrameID、Buffer.PixelFormat、Buffer.XOffset、Buffer.YOffset以及數據塊數據等更多有用的信息。如有必要，您還需復制如下信息。
pDevice->GetRemoteNode("PixelFormat")->SetString("Mono8");
BGAPI2::Buffer* pBufferFilled = pDataStream->GetFilledBuffer(1000);
if (pBufferFilled->GetPixelFormat() == "Mono8")
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC1,
(char *)pBufferFilled->GetMemPtr();
//3 methods to copy
cv::Mat imClone = imOriginal->clone();
cv::Mat* imCopy = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC1);
imOriginal->copyTo(*imCopy);
cv::Mat* imConvert = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC1);
imOriginal->convertTo(*imConvert, CV_8UC1, 1.0);
delete imOriginal;
pBufferFilled->QueueBuffer();
//use copied image
cv::imwrite("cv_image_Clone.png", imClone);
cv::imwrite("cv_image_Copy.png", *imCopy);
cv::imwrite("cv_image_Convert.png", *imConvert);
delete imCopy;
delete imConvert;
}
mDevice.RemoteNodeList["PixelFormat"].Value = "Mono8";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if (mBufferFilled.PixelFormat == "Mono8")
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U, 1,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 1);
//3 methods to copy
Emgu.CV.Mat imClone = imOriginal.Clone();
Emgu.CV.Mat imCopy = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U, 1);
imOriginal.CopyTo(imCopy);
Emgu.CV.Mat imConvert = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U, 1);
imOriginal.ConvertTo(imConvert, Emgu.CV.CvEnum.DepthType.Cv8U, 1.0);
mBufferFilled.QueueBuffer();
//use copied image
Emgu.CV.CvInvoke.Imwrite("cv_image_Clone.png", imClone);
Emgu.CV.CvInvoke.Imwrite("cv_image_Copy.png", imCopy);
Emgu.CV.CvInvoke.Imwrite("cv_image_Convert.png", imConvert);
}
關于像素格式
堡盟相機支持Mono8、Mono12、BGR8等多種像素格式（PixelFormat）——每種格式需對應相應的OpenCV格式。與堡盟相機像素格式對應的OpenCV矩陣類型如下表所示。
堡盟格式 | OpenCV格式 |
Mono8 | CV_8UC1 (8位, 1通道) |
Mono10 | 轉換為Mono16 (按位移位) |
Mono12 | 轉換為Mono16 (按位移位) |
Mono16 | CV_16UC1 (16位, 1通道) |
BGR8 | CV_8UC3 (8位, 3通道) |
BGR10 | 轉換為BGR16 (按位移位) |
BGR12 | 轉換為BGR16 (按位移位) |
BGR16 | CV_16UC3 (16位, 3通道 (BGR)) |
RGB8 | 轉換為BGR8 (cv::cvtColor) |
RGB10 | 轉換為BGR16 (按位移位和cv::cvtColor) |
RGB12 | 轉換為BGR16 (按位移位和cv::cvtColor) |
RGB16 | 轉換為BGR16 (cv::cvtColor) |
BayerGB8, BayerRG8, BayerGR8, BayerBG8 | 轉換為BGR8 (cv::cvtColor) |
BayerGB10, BayerRG10, BayerGR10, BayerBG10 | 轉換為BGR16 (按位移位和cv::cvtColor) |
BayerGB12, BayerRG12, BayerGR12, BayerBG12 | 轉換為BGR16 (按位移位和cv::cvtColor) |
從如下示例可以看出，可直接處理像素格式的圖像，無需轉換，非常簡單。
示例：Mono8
pDevice->GetRemoteNode("PixelFormat")->SetString("Mono8");
BGAPI2::Buffer * pBufferFilled = pDataStream->GetFilledBuffer(1000);
if (pBufferFilled->GetPixelFormat() == "Mono8")
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC1,
(char *)pBufferFilled->GetMemPtr();
cv::imwrite("cv_Mono8_image.png", *imOriginal);
delete imOriginal;
}
pBufferFilled->QueueBuffer();
mDevice.RemoteNodeList["PixelFormat"].Value = "Mono8";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if (mBufferFilled.PixelFormat == "Mono8")
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U, 1,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 1);
Emgu.CV.CvInvoke.Imwrite("cv_Mono8_image.png", imOriginal);
}
mBufferFilled.QueueBuffer();
示例：BGR8
OpenCV中標準的顏色像素格式為BGR，每個顏色通道為8位（CV_8UC3）。相機中與之對應的像素格式設為BGR8（也可以設為BGR8Packed）。
pDevice->GetRemoteNode("PixelFormat")->SetString("BGR8");
BGAPI2::Buffer * pBufferFilled = pDataStream->GetFilledBuffer(1000);
if ((pBufferFilled->GetPixelFormat() == "BGR8") ||
(pBufferFilled->GetPixelFormat() == "BGR8Packed"))
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC3,
(char *)pBufferFilled->GetMemPtr();
cv::imwrite("cv_BGR8_image.png", *imOriginal);
delete imOriginal;
}
pBufferFilled->QueueBuffer();
mDevice.RemoteNodeList["PixelFormat"].Value = "BGR8";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if ((mBufferFilled.PixelFormat == "BGR8") ||
(mBufferFilled.PixelFormat == "BGR8Packed"))
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U,
3,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 3);
Emgu.CV.CvInvoke.Imwrite("cv_BGR8_image.png", imOriginal);
}
mBufferFilled.QueueBuffer();
將像素格式轉換為16位OpenCV矩陣
某些相機支持12位像素格式（Mono12、RGB12）。將數據儲存在16位OpenCV矩陣類型中會導致輸出圖像比預期暗很多。為了將12位像素格式轉換成16位OpenCV矩陣，堡盟GAPI緩沖區中的數據需要通過OpenCV ConvertTo()函數平移4位（或乘以16）。
同樣地，Mono10等10位像素格式可以通過將數據平移2位（或乘以64）進行轉換。
警告：由于一個指針剛被傳輸到堡盟GAPI緩沖區的數據中，初始數據已經被更改。
pDevice->GetRemoteNode("PixelFormat")->SetString("Mono12");
BGAPI2::Buffer * pBufferFilled = pDataStream->GetFilledBuffer(1000);
if (pBufferFilled->GetPixelFormat() == "Mono12")
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_16UC1,
(char *)pBufferFilled->GetMemPtr();
imOriginal.ConvertTo(imOriginal, Emgu.CV.CvEnum.DepthType.Cv16U, 64.0);
cv::imwrite("cv_Mono12_as_Mono16_image.png", *imOriginal);
delete imOriginal;
}
pBufferFilled->QueueBuffer();
mDevice.RemoteNodeList["PixelFormat"].Value = "Mono12";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if (mBufferFilled.PixelFormat == "Mono12")
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv16U,
1,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 2);
imOriginal.ConvertTo(imOriginal, Emgu.CV.CvEnum.DepthType.Cv16U, 16.0);
Emgu.CV.CvInvoke.Imwrite("cv_Mono12_as_Mono16_image.png", imOriginal);
}
mBufferFilled.QueueBuffer();
Bayer像素格式
Bayer像素格式廣泛用于CCD和CMOS相機。采用Bayer格式，可以從一個單獨的平面得到彩色圖像，該平面上的R、G和B像素點（即特定分量的傳感器）交錯排列（如左圖所示）。
對于單個像素，輸出的RGB分量是由該像素周圍具有相同顏色的1個、2個或4個相鄰像素進行插值得到的。Bayer陣列可以通過向左和/或向上平移一個像素來進行修改。轉換常數CV_Bayer C1C2 2BGR和CV_Bayer C1C2 2RGB中的C1和C2兩個字母代表特定的陣列類型，分別是第二行、第二列和第三列的顏色分量。例如，陣列中有一個很常用的“BG”類型。
使用cvtColor的OpenCV顏色轉換
彩色相機支持BayerBG8、BayerGB8、BayerGR8和BayerRG8等原始像素格式。
如需進一步進行圖像處理，必須將這些格式轉換為BGR8格式。
堡盟相機根據圖像第一行的前兩個像素給Bayer陣列命名。而OpenCV則根據圖像第二行的第二和第三個像素給陣列命名，因此命名方式有所不同，對應關系如下：
堡盟相機的BayerGB對應OpenCV的BayerGR
堡盟相機的BayerRG對應OpenCV的BayerBG
堡盟相機的BayerGR對應OpenCV的BayerGB
堡盟相機的BayerBG對應OpenCV的BayerRG
pDevice->GetRemoteNode("PixelFormat")->SetString("BayerRG8");
BGAPI2::Buffer * pBufferFilled = pDataStream->GetFilledBuffer(1000);
if (pBufferFilled->GetPixelFormat() == "BayerRG8")
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC1,
(char *)pBufferFilled->GetMemPtr();
cv::Mat* imTransformBGR8 = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_8UC3);
//Baumer: RGrgrg >> OpenCV: rgrgrg
// gbgbgb gBGbgb
cv::cvtColor(*imOriginal, *imTransformBGR8, CV_BayerBG2BGR); //to BGR
delete imOriginal;
pBufferFilled->QueueBuffer();
cv::imwrite("cv_BayerRG8_as_BGR8_image.png", *imTransformBGR8);
delete imTransformBGR8;
}
mDevice.RemoteNodeList["PixelFormat"].Value = "BayerRG8";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if (mBufferFilled.PixelFormat == "BayerRG8")
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv8U, 1,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 1);
Emgu.CV.Mat imTransformBGR8 = new Emgu.CV.Mat();
//Baumer: RGrgrg >> OpenCV: rgrgrg
// gbgbgb gBGbgb
Emgu.CV.CvInvoke.CvtColor(imOriginal, imTransformBGR8,
Emgu.CV.CvEnum.ColorConversion.BayerBg2Bgr);
mBufferFilled.QueueBuffer();
Emgu.CV.CvInvoke.Imwrite("cv_BGR8_image.png", imTransformBGR8);
}
將Bayer格式轉換為BGR16
BayerBG12、BayerGB12、BayerGR12和BayerRG12等其他原始像素格式在Bayer圖像中使用更多位數。首先需要將這些原始數據從12位轉換為16位。之后，可以通過cvtColor將16位Bayer格式轉換為48位BGR16格式（每個顏色通道為16位）。
對于BayerBG10、BayerGB10、BayerGR10或BayerRG10等10位Bayer格式，在將其轉換為16位Bayer格式時，比例因子應選取64.0而非16.0。
pDevice->GetRemoteNode("PixelFormat")->SetString("BayerGB12");
BGAPI2::Buffer * pBufferFilled = pDataStream->GetFilledBuffer(1000);
if (pBufferFilled->GetPixelFormat() == "BayerGB12")
{
cv::Mat* imOriginal = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_16UC1,
(char *)pBufferFilled->GetMemPtr();
cv::Mat* imConvert = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_16UC1);
//convert with scaling of 16.0 to convert 12-Bit to 16-Bit
imOriginal->convertTo(*imConvert, CV_16UC1, 16.0);
delete imOriginal;
pBufferFilled->QueueBuffer();
cv::Mat* imTransformBGR16 = new cv::Mat((int)pBufferFilled->GetHeight(),
(int)pBufferFilled->GetWidth(),
CV_16UC3);
//Baumer: GBgbgb >> OpenCV: gbgbgb
// rgrgrg rGRgrg
cv::cvtColor(*imConvert, *imTransformBGR16, CV_BayerGR2BGR); //to BGR
delete imConvert;
cv::imwrite("cv_BayerGB12_as_BGR16_image.png", *imTransformBGR16);
delete imTransformBGR16;
}
mDevice.RemoteNodeList["PixelFormat"].Value = "BayerGB12";
BGAPI2.Buffer mBufferFilled = mDataStream.GetFilledBuffer(1000);
if (mBufferFilled.PixelFormat == "BayerGB12")
{
Emgu.CV.Mat imOriginal = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv16U, 1,
(IntPtr)mBufferFilled.MemPtr,
(int)mBufferFilled.Width * 2);
Emgu.CV.Mat imConvert = new Emgu.CV.Mat((int)mBufferFilled.Height,
(int)mBufferFilled.Width,
Emgu.CV.CvEnum.DepthType.Cv16U, 1);
//convert with scaling of 16.0 to convert 12-Bit to 16-Bit
imOriginal.ConvertTo(imConvert, Emgu.CV.CvEnum.DepthType.Cv16U, 16.0);
mBufferFilled.QueueBuffer();
Emgu.CV.Mat imTransformBGR16 = new Emgu.CV.Mat();
//Baumer: GBgbgb >> OpenCV: gbgbgb
// rgrgrg rGRgrg
Emgu.CV.CvInvoke.CvtColor(imConvert, imTransformBGR16,
Emgu.CV.CvEnum.ColorConversion.BayerGr2Bgr);
Emgu.CV.CvInvoke.Imwrite("cv_BayerGB12_as_BGR16_image.png", imTransformBGR16);
}
下載
軟件示例
-
堡盟GAPI和OpenCV